reproducing Kernel Hilbert Space (RKHS) reproducing property I'm a bit confused about the reproducing property of an RKHS, especially how a function $f$ say, is represented in the space. Suppose $\cal{X}$ is a set and $\cal{H}$ is a Hilbert space. Given a function $K: \cal{X} \times \cal{X} \rightarrow \Bbb{R}$, we say that $K$ has the reproducing property if $K(x, \cdot)$, i.e. $K$ considered as a function of its second argument, with the first fixed as $x$, also sometimes written as $K_{x}(\cdot)$, satisfies, for any function $f\in\cal{H}$:
$$
<f, K_x(\cdot)>_{\cal{H}} = f(x)
$$
So, the function $f$ can be represented as an inner product of functions in $\cal{H}$. But $f$ appears on both sides of the above equation, so what is the representation of $f$ on the LHS? For example if $x=[x_1, x_2]^T$ and $f(x) = ax_1 + bx_2 + cx_1x_2$, then $f$ can be represented by $[a, b, c]^T$ on the LHS, and $K_x(\cdot) = \phi(x) = [x_1, x_2, x_1x_2]^T$. So, in this example $f$ is viewed as both a function (of $x$) and as a vector of coefficients. What happens for more general forms of $f$? Can they always be written as a vector, and if so, what's the construction? Also, $K_x(\cdot)$ is not really a function of its second argument anymore, which it seems to have "lost". My background is theoretical CS, so I like to think in terms of types, which is probably what's confusing me here. Can someone help me with my misunderstanding?
 A: I think you are thinking in terms of finite-dimensional spaces.
I assume your space $\mathcal{H}$ is a space of functions on $\mathcal{X}$. By definition $f$ is a vector of your Hilbert space, but this space could well be infinite-dimensional and $f$ would have infinitely many coordinates (countable in the separable case). Think, for example, of the space $L^2[-\pi,\pi]$ where you have chosen the trigonometric basis. The dot product could well be an integral like
$$
\langle f,g\rangle=\int\limits_{[-\pi,\pi]} f(x) g(x)\,dx
$$
which becomes a series when expressed in coordinates (say, i the trigonometric basis). 
The language of matrices is not very appropriate in those cases, and thinking of the elements of the space as row vectors is not useful.
The function $K_x(\cdot)$ is just a function of $\mathcal{H}$, so it makes perfect sense to consider its dot product with some other $f\in\mathcal{H}$. The whole point is that the real number you obtain is precisely the value of $f(x)$. 
Not sure if I am giving a satisfactory answer. Let me know otherwise.
