I'm a bit confused about the reproducing property of an RKHS, especially how a function $f$ say, is represented in the space. Suppose $\cal{X}$ is a set and $\cal{H}$ is a Hilbert space. Given a function $K: \cal{X} \times \cal{X} \rightarrow \Bbb{R}$, we say that $K$ has the reproducing property if $K(x, \cdot)$, i.e. $K$ considered as a function of its second argument, with the first fixed as $x$, also sometimes written as $K_{x}(\cdot)$, satisfies, for any function $f\in\cal{H}$:

$$ <f, K_x(\cdot)>_{\cal{H}} = f(x) $$

So, the function $f$ can be represented as an inner product of functions in $\cal{H}$. But $f$ appears on both sides of the above equation, so what is the representation of $f$ on the LHS? For example if $x=[x_1, x_2]^T$ and $f(x) = ax_1 + bx_2 + cx_1x_2$, then $f$ can be represented by $[a, b, c]^T$ on the LHS, and $K_x(\cdot) = \phi(x) = [x_1, x_2, x_1x_2]^T$. So, in this example $f$ is viewed as both a function (of $x$) and as a vector of coefficients. What happens for more general forms of $f$? Can they always be written as a vector, and if so, what's the construction? Also, $K_x(\cdot)$ is not really a function of its second argument anymore, which it seems to have "lost". My background is theoretical CS, so I like to think in terms of types, which is probably what's confusing me here. Can someone help me with my misunderstanding?


1 Answer 1


I think you are thinking in terms of finite-dimensional spaces.

I assume your space $\mathcal{H}$ is a space of functions on $\mathcal{X}$. By definition $f$ is a vector of your Hilbert space, but this space could well be infinite-dimensional and $f$ would have infinitely many coordinates (countable in the separable case). Think, for example, of the space $L^2[-\pi,\pi]$ where you have chosen the trigonometric basis. The dot product could well be an integral like $$ \langle f,g\rangle=\int\limits_{[-\pi,\pi]} f(x) g(x)\,dx $$ which becomes a series when expressed in coordinates (say, i the trigonometric basis).

The language of matrices is not very appropriate in those cases, and thinking of the elements of the space as row vectors is not useful.

The function $K_x(\cdot)$ is just a function of $\mathcal{H}$, so it makes perfect sense to consider its dot product with some other $f\in\mathcal{H}$. The whole point is that the real number you obtain is precisely the value of $f(x)$.

Not sure if I am giving a satisfactory answer. Let me know otherwise.

  • $\begingroup$ Good answer. It'd be great if you provided an example where the evaluation of f equals the inner product of K and f. $\endgroup$
    – xabush
    Nov 12, 2021 at 15:30
  • $\begingroup$ You might find it useful to look at the proof of the Riesz Representation Theorem. If an example is a means toward certainty, proof is the certainty. (By the way, $L^2$ is a slightly unfortunate choice here, since it isn't a RKHS. However, it demonstrates the point.) $\endgroup$
    – demim00nde
    Mar 1, 2023 at 18:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .