I am trying to better understand Mercer's Theorem, by applying it to some specific kernels.


Let $D \subset \mathbb{R}^N$ be a closed, bounded subset. We associate a function $K: D \times D \rightarrow \mathbb{R}$ with an operator (the Hilbert-Schmidt Integral Operator) $T_K: L^2(D) \rightarrow L^2(D)$ being $$ [T_K\phi](x) = \int_{D} K(x,s) \phi(s) \, ds. $$ We say that $K$ is positive semi-definite if for every set of $m$ vectors $(x_i)_{i \in \{ 1,2,\ldots,m\}} \in D$, the corresponding Gram matrix $[K(x_i, x_j)]_{ij}$ is positive semi-definite.

Mercer's Theorem states that if $K$ is symmetric, positive semi-definite, then it holds that $$ K(s,t) = \sum_{j = 1}^{\infty} \lambda_j e_j(s)e_j(t), \quad \quad ... (1) $$ where $\lambda_j$ is the $j$-th eigenvalue of $T_K$, and $e_j$ is the corresponding eigenfunction.


While I understand this statement in an abstract sense, I have a hard time working through the theorem with an actual kernel. Take two simple examples:

  • $K_1(x,y) = x^{\intercal}y$
  • $K_2(x,y) = (x^{\intercal}y)^2$

To be clear, $x, y$ are vectors in $\mathbb{R}^N$. It's easily checked that $K_1, K_2$ are symmetric and positive semi-definite. So in this case, Mercer's Theorem should be applicable. But I have trouble determining what exactly $T_{K_1}$ or $T_{K_2}$ are, and furthermore what the corresponding $\lambda_j, e_j$ are so that (1) will hold.

Can someone please enlighten me? Thanks.

Edit (some clarifications):

To see that $K_1$ is positive semi-definite, fix $(x_i)_{i \in \{ 1,2,\ldots,m\}} \in D$, and denote the matrix $X$ whose $i$-th column is $x_i$. Then $[K_1(x_i, x_j)]_{ij} = X^T X$. Thus, for any $z \in \mathbb{R}^N$, we have $$ z^T X^T X z = \| Xz\|^2 \geq 0, $$ proving positive semi-definiteness.

  • $\begingroup$ Could you explain why $x^Ty$ is positive semi-definite? I would think that $y=-x$ will result in a negative quantity $x^T(-x)=-x^Tx<0 \quad \forall x\in \mathbb{R}^{n}-\{x \neq 0 \}$. What is the Gram matrix? $\endgroup$ – MrYouMath Feb 28 '18 at 8:24
  • 1
    $\begingroup$ I have added some clarification. I hope that helps. $\endgroup$ – Josh Feb 28 '18 at 8:33

Here is a solution for $N=1$ and $D=[a,b]$.

Note that in both cases, the kernel can be factorized as $K(x,y)=f(x)f(y)$. Thus the eigenvalue equation simplifies to $$ \lambda \phi(x)=f(x)\int_a^b f(y)\phi(y)\,dy. $$ Hence, if $\lambda\neq 0$ (and this is the interesting case), then $\phi(x)=\alpha f(x)$ for some $\alpha\in \mathbb{R}$ and consequently $$ \lambda=\int_a^b f(x)^2\,dx. $$ To get a normalized eigenfunction, we must have $$ 1=\int_a^b \alpha^2 f(x)^2\,dx, $$ that is, $\alpha=\left(\int_a^b f(x)^2\,dx\right)^{-1/2}$. It follows that $$ \lambda \phi(x)\phi(y)=\left(\int_a^b f(x)^2\,dx\right)\left(\int_a^b f(x)^2\,dx\right)^{-1}f(x)f(y)=K(x,y), $$ as was expected from Mercer's theorem.

In the case of $K_1$ this gives the (only) non-zero eigenvalue $\lambda=\frac 1 3(b^3-a^3)$ with corresponding eigenfunction $\phi(x)=\lambda^{-1/2}x$, for $K_2$ we get $\lambda=\frac 1 5(b^5-a^5)$ and $\phi(x)=\lambda^{-1/2}x^2$.

In higher dimension, at least if $D=[a_1,b_1]\times\dots\times[a_N,b_N]$, one can still decompose $K$ as $K(x,y)=f_1(x)g_1(y)+\dots+f_n(x)g_n(y)$ and try a similar approach, but things get messier (I don't know if you get explicit results, haven't spelled it out).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.