2
$\begingroup$

Let the input space be $\mathcal{X}=\mathbb{R}^2$, and kernel function $k(x,y)=\langle x,y \rangle^2 = (x_1y_1+x_2y_2)^2$.

I can write

\begin{align} k(x,y) &= x_1^2 y_1^2 + x_2^2 y_2^2 + 2 x_1 y_1 x_2 y_2 \\ &= \begin{bmatrix} x_1^2 & x_2^2 & \sqrt{2}x_1 x_2 \end{bmatrix} \begin{bmatrix} y_1^2 \\ y_2^2 \\ \sqrt{2}y_1 y_2 \end{bmatrix}\\ \end{align}

The feature map $\phi: \mathbb{R}^2 \to \mathbb{R}^3$ is then $\phi(x)=k_x=k(\cdot, x) = (x_1^2, x_2^2, \sqrt{2}x_1 x_2)$. Since $\phi: \mathcal{X} \to \mathcal{H}$ maps input vector to the RKHS defined by $k$, I conclude that $\mathcal{H} = \mathbb{R}^3$.

But page 41 of this lecture presentation says otherwise, i.e. $\mathbb{R}^3$ is not the RKHS defined by kernel $k(x,y)=(x_1y_1+x_2y_2)^2$. If not, then what is the RKHS defined by kernel $k$?

$\endgroup$

1 Answer 1

Reset to default
-1
$\begingroup$

The RKHS defined by that kernel is not the "whole" $\mathbb{R}^3$, but a two dimensional manifold defined in it. Note that, for instance, the vector $(1,1,1)$ is not in the image of $\phi(x)$.

$\endgroup$
3
  • 1
    $\begingroup$ My impression is that $\mathcal{H}$ is not defined by $\{\phi(x) | x \in \mathcal{X} \}$, but all functions of the form $f(x)=ax_1^2+bx_2^2+cx_1x_2$ (hence $\mathcal{H}=\mathbb{R}^3$) . See these slides, p.36-37 $\endgroup$
    – Yibo Yang
    Mar 12, 2017 at 0:11
  • $\begingroup$ The kernel which is reproducing on some Hilbert space does map into a manifold within that space, but is a reproducing kernel on the entire space. Remember that the space is defined as the completion of the space spanned by $K(., x)_{x\in\mathbb{R}^2}$ (c.f. Moore-Aronszajn Theorem). $\endgroup$
    – Marko Stojovic
    Mar 1 at 19:06
  • $\begingroup$ On the other hand, I'm still trying to get my head around which space is the unique RKHS associated with a given kernel, given that one could similarly construct a mapping into $\mathbb{R}^4$, or indeed an alternative mapping into $\mathbb{R}^3$ (see the 1998 tutorial on support vector machines by Burgess). I have a running theory that the mapping is in fact into $l^2({\mathbb{R^2}})$, and we treat any finite-dimensional Euclidian space as a Hilbert subspace (this doesn't then violate uniqueness of the RKHS space) but I'm still trying to understand it. Thoughts welcome! $\endgroup$
    – Marko Stojovic
    Mar 1 at 19:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .