# reproducing Kernel Hilbert Space (RKHS) reproducing property

I'm a bit confused about the reproducing property of an RKHS, especially how a function $$f$$ say, is represented in the space. Suppose $$\cal{X}$$ is a set and $$\cal{H}$$ is a Hilbert space. Given a function $$K: \cal{X} \times \cal{X} \rightarrow \Bbb{R}$$, we say that $$K$$ has the reproducing property if $$K(x, \cdot)$$, i.e. $$K$$ considered as a function of its second argument, with the first fixed as $$x$$, also sometimes written as $$K_{x}(\cdot)$$, satisfies, for any function $$f\in\cal{H}$$:

$$_{\cal{H}} = f(x)$$

So, the function $$f$$ can be represented as an inner product of functions in $$\cal{H}$$. But $$f$$ appears on both sides of the above equation, so what is the representation of $$f$$ on the LHS? For example if $$x=[x_1, x_2]^T$$ and $$f(x) = ax_1 + bx_2 + cx_1x_2$$, then $$f$$ can be represented by $$[a, b, c]^T$$ on the LHS, and $$K_x(\cdot) = \phi(x) = [x_1, x_2, x_1x_2]^T$$. So, in this example $$f$$ is viewed as both a function (of $$x$$) and as a vector of coefficients. What happens for more general forms of $$f$$? Can they always be written as a vector, and if so, what's the construction? Also, $$K_x(\cdot)$$ is not really a function of its second argument anymore, which it seems to have "lost". My background is theoretical CS, so I like to think in terms of types, which is probably what's confusing me here. Can someone help me with my misunderstanding?

I assume your space $$\mathcal{H}$$ is a space of functions on $$\mathcal{X}$$. By definition $$f$$ is a vector of your Hilbert space, but this space could well be infinite-dimensional and $$f$$ would have infinitely many coordinates (countable in the separable case). Think, for example, of the space $$L^2[-\pi,\pi]$$ where you have chosen the trigonometric basis. The dot product could well be an integral like $$\langle f,g\rangle=\int\limits_{[-\pi,\pi]} f(x) g(x)\,dx$$ which becomes a series when expressed in coordinates (say, i the trigonometric basis).
The function $$K_x(\cdot)$$ is just a function of $$\mathcal{H}$$, so it makes perfect sense to consider its dot product with some other $$f\in\mathcal{H}$$. The whole point is that the real number you obtain is precisely the value of $$f(x)$$.
• You might find it useful to look at the proof of the Riesz Representation Theorem. If an example is a means toward certainty, proof is the certainty. (By the way, $L^2$ is a slightly unfortunate choice here, since it isn't a RKHS. However, it demonstrates the point.) Mar 1, 2023 at 18:56