# Proving that convergence in RKHS implies pointwise convergence without using reproducing property

Let $$(\mathcal{H}, \mathcal{K})$$ be a reproducing kernel Hilbert space and denote $$\mathcal{K}_x := \mathcal{K}(x, \cdot)$$.
Is there a simple way to prove $$f_n \to_\mathcal{H} f$$ (shorthand for $$\|f_n - f\|_\mathcal{H} \to 0$$) implies $$f_n \to f$$ pointwise without having to invoke the reproducing property $$\langle g, \mathcal{K}_x \rangle_\mathcal{H} = g(x)$$?

I'm likely misunderstanding something, but it appears to me that this result is needed to prove the reproducing property itself for functions not in $$\mathcal{S} := \text{span}\{\mathcal{K}_x | x \in \mathcal{X}\}$$ but in its completion when constructing the RKHS $$\mathcal{H}$$.
It is easy to see that any function in $$\mathcal{S}$$ satisfies the reproducing property.
For $$f \in \mathcal{H} \setminus \mathcal{S}$$, by definition we can always find a sequence $$f_n$$ in $$\mathcal{S}$$ such that $$f_n \to_\mathcal{H} f$$. Then,

\begin{align*} \langle f, \mathcal{K}_x \rangle_\mathcal{H} &= \lim_{n \to \infty} \langle f_n, \mathcal{K}_x \rangle_\mathcal{H} \quad \text{since } \langle \cdot, \mathcal{K}_x \rangle_\mathcal{H} \text{ is continuous w.r.t } \| \cdot \|_\mathcal{H} \\ &= \lim_{n \to \infty} f_n(x) \quad \text{by the reproducing property for functions in } \mathcal{S} \\ &= f(x) \quad \text{assuming we can use the fact that } f_n \to_\mathcal{H} f \text{ implies } f_n \to f. \end{align*}

Edit:
To be clear, I'm following the proof of Theorem 12.11 from Wainwright's High-dimensional statistics, which states that

For any positive semidefinite kernel $$\mathcal{K}$$, there is a unique Hilbert space $$\mathcal{H} \subset \mathbb{R}^\mathcal{X}$$ in which the kernel satisfies the reproducing property: $$\langle f, \mathcal{K}_x \rangle_\mathcal{H} = f(x) \quad \forall f \in \mathcal{H}.$$ $$\mathcal{H}$$ is known as the RKHS associated with $$\mathcal{K}$$.

Therefore, instead of defining the RKHS to be a space of bounded evaluation functionals (this appears as a theorem in the book), the goal is to construct the RKHS by completing the span of $$\{\mathcal{K}_x\}_{x \in \mathcal{X}}$$ and show that it satisfies all the required properties.

• Isn't this guaranteed by the definition of RKHS? i.e., the functional $f \mapsto f(x)$ is linear and continous.
– daw
Nov 30, 2023 at 9:28
• @daw Thanks for the comment! Can you clarify what this entails? I know this can be used in conjunction with the reproducing property to prove that RKHS convergence implies pointwise convergence, but this creates a tautology if I then want to apply this result to prove the reproducing property. Nov 30, 2023 at 23:20
• The definition of RKHS on wikipedia is that the evaluation functionals $f\mapsto f(x)$ are continuous wrt to the RKHS-norm. Then $f_n \to f$ in $H$ implies $f_n(x)\to f(x)$ for all $x$.
– daw
Dec 1, 2023 at 7:04
• @daw Ah I realize where the confusion is - I'm starting from a different definition of RKHS. Instead of defining it to be the space of continuous/bounded evaluation functionals, the book I'm following defines it to be the unique Hilbert space that satisfies the reproducing property. I'll update the post to be more clear. Dec 1, 2023 at 23:47

As mentioned by @daw, for a Hilbert space $$\mathcal{H}$$ to become a RKHS the evaluation functionals $$L_{x}​(f):=f(x)$$ must be continuous, which is equivalent to being bounded. Thus for every $$x$$ there is a $$M_{x}$$ with $$L_{x}(f)=f(x)\leq M_{x} \| f \|_{\mathcal{H}}$$.
It follows that $$f_{n} \rightarrow f$$ implies the pointwise convergence. Pick an arbitrary $$x$$ then $$|f(x)-f_{n}(x)|=|(f-f_{n})(x)|=|L_{x}(f-f_{n})| \leq M_{x}\|f-f_{n}\|$$ and since $$\|f_{n} - f\| \rightarrow 0$$ we see that $$f_{n}(x) \rightarrow f(x)$$.
• I think in some way or another you will need to use this property since this is what distinguishes a RKHS from a Hilbert space. And in a Hilbert space your implication does not hold as can be seen f.e. by considering $L^{2}$. Dec 2, 2023 at 16:18
• I think what you are looking for is he Moore-Aronszajn theorem. As you can see, the reproducing property arises from the definition of the inner product on $\mathcal{S}$. Dec 4, 2023 at 15:25