I'm following the gentle introduction to Reproducing Kernel Hilbert Spaces From Zero to Reproducing Kernel Hilbert Spaces in Twelve Pages or Less by Hal Daum'e III. I believe the author fully intended to hand-wave in several places, and that's been mostly fine for my purposes since I've been able to extract the main idea and fill in some of the details, but I do not have a slight idea about the following.

We pick it up in Section 6.4 on page 9 and follow the author's notation. Given a positive definite kernel $K$ over $X$, we wish to construct an RKHS $\mathcal{H}$ of functions $f : X \to \mathbb{R}$. For this construction, we need an inner product, so the author writes the obvious thing

$$ \langle k_x , k_y \rangle_{\mathcal{H}_K} = \Big\langle \sum_i \alpha_i k_{x_{i}}, \sum_i \beta_i k_{y_{i}} \Big\rangle_{X} $$

since we have already added the span of $\{k_x\}_{x\in X}$ to the RKHS under construction.

One page later, the author appeals to Mercer-Hilbert-Schmidt theorems and Fourier analysis to argue that the dot product can be defined by

$$ \langle y, y^{\prime}\rangle_{\mathcal{H}_{K}} = \sum_{i=0}^{\infty} \dfrac{y_i y_{i}^{\prime}}{\lambda_i} $$

where $\lambda_i$ are eigenvalues and

$$ y_i := \int y(x) \phi_i (x) dx $$

for eigenfunctions $\phi$ (the decision to switch to $y\in \mathcal{H}$ was the author's).

My question is NOT the derivation of this, but why it is needed. What's so wrong with the first inner product we defined that forces us to come up with this second representation based on eigenfunctions? Thanks.


1 Answer 1


I didn't read the whole text , but it seems that the first expression holds only for some restricted set of funtions $k_{x}$ (they are defined by $k_{x}(y) = K(x,y)$) , while the second is the general definition of a inner product in the Hilbert space .


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .