A New Upper Bound for Sampling Numbers

Foundations of Computational Mathematics https://doi.org/10.1007/s10208-021-09504-0 A New Upper Bound for Sampling Numbers Nicolas Nagel1 · Martin Schäfer1 · Tino Ullrich1 Received: 18 October 2020 / Revised: 31 December 2020 / Accepted: 18 January 2021 © The Author(s) 2021 Abstract We provide a new upper bound for sampling numbers (gn )n∈N associated with the compact embedding of a separable reproducing kernel Hilbert space into the space of square integrable functions. There are universal constants C, c > 0 (which are specified in the paper) such that gn2 ≤ C log(n)  2 σk , n ≥ 2, n k≥cn where (σk )k∈N is the sequence of singular numbers (approximation numbers) of the Hilbert–Schmidt embedding Id : H (K ) → L 2 (D,  D ). The algorithm which realizes the bound is a least squares algorithm based on a specific set of sampling nodes. These are constructed out of a random draw in combination with a down-sampling procedure coming from the celebrated proof of Weaver’s conjecture, which was shown to be equivalent to the Kadison–Singer problem. Our result is non-constructive since we only show the existence of a linear sampling operator realizing the above bound. s (Td ) The general result can for instance be applied to the well-known situation of Hmix d in L 2 (T ) with s > 1/2. We obtain the asymptotic bound gn ≤ Cs,d n −s log(n)(d−1)s+1/2 , which improves on very recent results by shortening the gap between upper and lower  bound to log(n). The result implies that for dimensions d > 2 any sparse grid sampling recovery method does not perform asymptotically optimal. Keywords Sampling recovery · Least squares approximation · Random sampling · Weaver’s conjecture · Finite frames · Kadison–Singer problem Communicated by Frances Kuo. B Tino Ullrich 1 Faculty of Mathematics, TU Chemnitz, 09107 Chemnitz, Germany 123 Foundations of Computational Mathematics Mathematics Subject Classification 41A25 · 41A63 · 68Q25 · 65Y20 1 Introduction In this paper, we study a well-known problem on the optimal recovery of multivariate functions from n function samples. The problem turned out to be rather difficult in several relevant situations. Since we want to recover the function f from n function samples ( f (x1 ), . . . , f (xn )), the problem boils down to the question of how to choose these sampling nodes X = (x1 , . . . , xn ) and corresponding recovery algorithms. The minimal worst-case error for an optimal choice is reflected by the n-th sampling number defined by gn (Id K , D ):= inf inf sup x1 ,...,xn ∈D ϕ:Cn →L 2  f  H (K ) ≤1  f − ϕ( f (x1 ), . . . , f (xn )) L 2 (D, D ) . (1) The functions are modeled as elements from a separable reproducing kernel Hilbert space H (K ) of functions on a set D ⊂ Rd with finite trace kernel K (·, ·), i.e.,  K (x, x)d D (x) < ∞. tr(K ):= (2) D The recovery problem (in the above framework) has been first addressed by Wasilkowski and Woźniakowski in [42]. The corresponding problem for certain particular cases (e.g., classes of functions with mixed smoothness properties, see [7, Sect. 5]) has been studied much earlier. Our main result is the existence of two universal constants C, c > 0 (specified in Remark 6.3) such that the relation gn2 ≤ C log(n)  2 σk , n ≥ 2, n (3) k≥cn holds true between the sampling numbers (gn )n∈N and the square summable singular numbers (σk )k∈N of the compact embedding Id K , D : H (K ) → L 2 (D,  D ). We emphasize that in general, the square-summability of the singular numbers (σk )k∈N is not implied by the compactness of the embedding Id K , D . This is one reason why we need the additional assumption of a finite trace kernel (2) (or a Hilbert–Schmidt embedding). In addition, as it has been observed by Hinrichs et al. [10], the non-existing trace may cause the sampling numbers to have a worse (or even no) polynomial decay than the corresponding polynomially decaying singular numbers (σk )k . Hence, an inequality (like (3)) which passes on the polynomial decay of the singular numbers to the sampling numbers is in general impossible without the condition of a finite trace (2). In our main example, the recovery of multivariate functions with dominating mixed smoothness (see Sect. 7), this condition is equivalent to s > 1/2, where s denotes 123 Foundations of Computational Mathematics the mixed smoothness parameter. For further historical and technical comments (e.g., non-separable RKHS) we refer to Remark 6.2. The algorithm which realizes the bound (3) in the sense of (1) is a (linear) least squares algorithm based on a specific set of sampling nodes. These are constructed out of a random draw in combination with a down-sampling procedure coming from the proof of Weaver’s conjecture [26], see Sect. 2. In its original form, the result in [26] is not applicable for our purpose. That is why we have to slightly generalize it, see Theorem 2.3. Note that the result in (3) is non-constructive. We do not have a deterministic construction for a suitable set of nodes. However, we have control of the failure probability which can be made arbitrarily small. In addition, the subspace, where the least squares algorithm is taking place is precisely given and determined by the first m singular vectors. The problem discussed in the present paper is tightly related to the problem of the Marcinkiewicz discretization of L 2 -norms for functions from finite-dimensional spaces (e.g., trigonometric polynomials). In fact, constructing well-conditioned matrices for the least squares approximation is an equivalent issue. Let us emphasize that V.N. Temlyakov (and coauthors) already used the Nitzan et al. construction [26] for the Marcinkiewicz discretization problem in the context of multivariate (hyperbolic cross) polynomials, see [34,35] and the very recent paper [21]. Compared to the result by Krieg and M.Ullrich [15], the relation (3) is stronger. In fact, the difference is mostly in the log-exponent as the example below shows. The general relation (3) yields a significant improvement in the situation of mixed Sobolev s (Td ) in embeddings in L 2 , see Sect. 7. Applied, for instance, to the situation of Hmix d L 2 (T ) with s > 1/2 (this condition is equivalent to the finite trace condition (2)), the result in (3) yields (4) gn d n −s log(n)(d−1)s+1/2 , whereas the result in [15] (see also [13,24,41]) implies gn d n −s log(n)(d−1)s+s . Thelog-gap grows with s > 1/2. Our new result achieves rates that are only worse by log(n) in comparison with the benchmark rates given by the singular numbers. Note that in d ≥ 3 and any s > 1/2 the bound (4) yields a better performance than any sparse grid technique is able to provide, see [2,3,6,8,31,33] and [7, Sect. 5]. In addition, combining the above result with recent preasymptotic estimates for the (σ j ) j , see [14,17–19], we are able to obtain reasonable bounds for gn also in the case of small n. See Sect. 7 for further comments and references in this (...truncated)