I am reading the book Random Fields and Geometry by R. Adler and J. Taylor. In chapter 3 they introduce the concept of reproducing kernel Hilbert space (RKHS). I will summarize the definition in what follows.

Firstly let us consider a Gaussian process $f$ defined on $T$ a metric space which we assume compact in the canonical metric $d$ generated by the process, where $$ d(s, t) = \sqrt{\mathbb{E}[(f(s)-f(t))^2]} $$ for $s,t \in T$. Let $C(s,t)$ be the covariance function of $f$ and let $C$ be continuous and positive definite. Let us define $$ S = \left\{ u \colon \ T \rightarrow \mathbb{R} \colon u(\cdot) = \sum_{n=1}^n a_i C (s_i, \cdot), a_i \in \mathbb{R}, s_i \in T, n \geq 1 \right \}. $$ Define an inner product on $S$ by $$ (u,v)_H = \left( \sum_{i=1}^n a_i C(s_i, \cdot), \sum_{j=1}^m b_j C(t_j, \cdot)\right)_H. $$ Note that, since we assumed that $C$ is positive definite $(\cdot, \cdot)_H$ induces a norm. Moreover the reproducing kernel property holds, namely $$ (u, C(t, \cdot)_H = \left( \sum_{i=1}^n a_i C(s_i, \cdot), C(t,\cdot)\right)_H = \sum_{i=1}^n a_i C(s_i, t)=u(t). $$ Let us define $H(C)$ the reproducing kernel hilbert space of $f$ the closure of $S$ under the norm induced by the inner product $( \cdot, \cdot)_H$.

The the authors state that since $T$ is separable (it is a compact metric space) then $H(C)$ is separable due to the continuity of $C$, how does if follows? My first try was: let $\{\phi_n\}_{n \in \mathbb{N}}$ be the orthonormal sequence on $T$, then I would like to say that $\psi_n(\cdot) = C (\phi_n, \cdot)$ is an orthonormal sequence on $H(C)$, but here I stop. Moreover is the inner product unique?


From the reproducing property of the RKHS it follows that the canonical feature map $\Phi:T \to H(C):t \mapsto C(\cdot ,t)$ is continuous, since if $t_n \to t$ in $T$, then $$ ||C(\cdot,t_n)-C(\cdot,t)||_H^2 = \underbrace{(C(\cdot,t_n),C(\cdot,t_n))_H}_{C(t_n,t_n)}- 2\underbrace{(C(\cdot,t_n),C(\cdot,t))_H}_{C(t_n,t)} + \underbrace{(C(\cdot,t),C(\cdot,t))_H}_{C(t,t)} = d(t_n,t)^2 \to 0\,. $$ But by definition of $H(C)$ we have $H(C) = \overline{\text{span}}\,\Phi(T)$ and since $T$ is separable and $\Phi$ is continuous, the set $\Phi(T)$ is separable and hence also $H(C)$.

One can show that actually $H(C)$ is really a space of functions, i.e. it can be made to consist of true functions $T \to \mathbb{R}$ and not only equivalence classes of sequences.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.