1
$\begingroup$

I see a lot of texts describing properties of Reproducing Kernel Hilbert Spaces (RKHS), but I can't seem to find one that explains how to actually calculate the norm in the RKHS. From Wikipedia, it says:

A RKHS is a space of functions in which point evaluation is a continuous linear functional. Roughly speaking, this means that if two functions $f$ and $g$ in the RKHS are close in norm, i.e., $\|f-g\|$ is small, then $f$ and $g$ are also pointwise close, i.e., $ |f(x)-g(x)|$ is small for all $x$. The reverse need not be true.

Questions:

  1. It seems a RKHS is a space full of functions, so each element in it is a function? But I can treat these elements as points nonetheless?
  2. How does the concept of "norm" make sense in this context? A Euclidean norm is the distance between two points in space, but this is the distance between two functions in a space?
  3. Assuming I understand question 2, how can I go about actually calculating this norm? An example or formula would be most appreciated.
$\endgroup$
3
  • 3
    $\begingroup$ For your second question, note that to define a RKHS, you are starting with a Hilbert space. A Hilbert space comes equipped with an inner product. The norm is then usually defined to be the positive square root of the inner product. As for your third question, how one may explicitly calculate the norm of a function in a RKHS, the definition of the norm depends on the definition of the inner product, like I mentioned. $\endgroup$ – tangentbundle Apr 7 '20 at 0:28
  • $\begingroup$ Also, this may be useful: math.stackexchange.com/questions/966715/… $\endgroup$ – tangentbundle Apr 7 '20 at 0:30
  • $\begingroup$ @tangentbundle thanks, that helps somewhat. Perhaps at this point a full fleshed out example might what I need to connect all the pieces. $\endgroup$ – guy Apr 7 '20 at 2:53
2
+50
$\begingroup$

a RKHS is a space full of functions, so each element in it is a function?

Yes.

I can treat these elements as points nonetheless?

Yes. RKHS can be considered as a metric or topological space, and elements of such spaces usually are considered as points. This often provides helpful geometrical visualizations. But, moreover, RKHS is also a linear and normed space, and elements of such spaces usually are considered as vectors. This also provides sometimes even more helpful and concrete geometrical visualizations.

How does the concept of "norm" make sense in this context? A Euclidean norm is the distance between two points in space, but this is the distance between two functions in a space?

We are going opposite direction in this case. A Hilbert space $H$ has an inner product $\langle\cdot,\cdot \rangle_H$ which provides a norm $\|\cdot\|_H$ on it. For a real Hilbert space we have $\|f\|_H=\sqrt{\langle f,f \rangle_H}$ for each $f\in H$. So $H$ can be considered as a normed space endowed with a norm $\|\cdot\|_H$. It provides a distance $d_H$ on $H$ such that $d_H(x,y)=\|f-g\|_H$ for each $f,g\in H$.

how can I go about actually calculating this norm? An example or formula would be most appreciated.

This norm is provided with a Hilbert space.

A basic example is a generalization of a finitely-dimensional Euclidean space $\Bbb R^n$, namely a linear space $\ell_2$ of functions $f$ from $\Bbb N$ to $\Bbb R$ such that $\sum_{n\in\Bbb N} f(n)^2<\infty$. It is endowed with the inner product $\langle f, g \rangle=\sum_{n\in\Bbb N} f(n)g(n)$ for each $f,g\in\ell_2$. Then $\ell_2$ is a normed space with a norm $\|f\|=\sqrt{\sum_{n\in\Bbb N} f(n)^2}$ for each $f\in\ell_2 $ and a metric space with a metric $d(f,g)=\sqrt{\sum_{n\in\Bbb N} (f(n)-g(n))^2}$. Finally, $\ell_2$ is RKHS because for each $n\in\Bbb n$ an evaluation operator $L_n:\ell_2\to\Bbb R$, $f\mapsto f(n)$ equals to an inner product $\langle f, e_n\rangle$ of the function $f$ with the standard unit vector $e_n\in\ell_2$, such that $e_n(n)=1$ and $e_n(m)=0$ for each natural $m\ne n$.

Remark that in the previous paragraph the set $N$ can be replaced by any set $X$, providing a Hilbert space $\ell_2(X)$. When $X$ is finite then the spaces $\ell_2(X)$ are isomorphic to finitely-dimensional Euclidean spaces $\Bbb R^n$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.