# Mercer's Theorem for some simple kernels

I am trying to better understand Mercer's Theorem, by applying it to some specific kernels.

## Background

Let $D \subset \mathbb{R}^N$ be a closed, bounded subset. We associate a function $K: D \times D \rightarrow \mathbb{R}$ with an operator (the Hilbert-Schmidt Integral Operator) $T_K: L^2(D) \rightarrow L^2(D)$ being $$[T_K\phi](x) = \int_{D} K(x,s) \phi(s) \, ds.$$ We say that $K$ is positive semi-definite if for every set of $m$ vectors $(x_i)_{i \in \{ 1,2,\ldots,m\}} \in D$, the corresponding Gram matrix $[K(x_i, x_j)]_{ij}$ is positive semi-definite.

Mercer's Theorem states that if $K$ is symmetric, positive semi-definite, then it holds that $$K(s,t) = \sum_{j = 1}^{\infty} \lambda_j e_j(s)e_j(t), \quad \quad ... (1)$$ where $\lambda_j$ is the $j$-th eigenvalue of $T_K$, and $e_j$ is the corresponding eigenfunction.

## Question

While I understand this statement in an abstract sense, I have a hard time working through the theorem with an actual kernel. Take two simple examples:

• $K_1(x,y) = x^{\intercal}y$
• $K_2(x,y) = (x^{\intercal}y)^2$

To be clear, $x, y$ are vectors in $\mathbb{R}^N$. It's easily checked that $K_1, K_2$ are symmetric and positive semi-definite. So in this case, Mercer's Theorem should be applicable. But I have trouble determining what exactly $T_{K_1}$ or $T_{K_2}$ are, and furthermore what the corresponding $\lambda_j, e_j$ are so that (1) will hold.

Can someone please enlighten me? Thanks.

Edit (some clarifications):

To see that $K_1$ is positive semi-definite, fix $(x_i)_{i \in \{ 1,2,\ldots,m\}} \in D$, and denote the matrix $X$ whose $i$-th column is $x_i$. Then $[K_1(x_i, x_j)]_{ij} = X^T X$. Thus, for any $z \in \mathbb{R}^N$, we have $$z^T X^T X z = \| Xz\|^2 \geq 0,$$ proving positive semi-definiteness.

• Could you explain why $x^Ty$ is positive semi-definite? I would think that $y=-x$ will result in a negative quantity $x^T(-x)=-x^Tx<0 \quad \forall x\in \mathbb{R}^{n}-\{x \neq 0 \}$. What is the Gram matrix? – MrYouMath Feb 28 '18 at 8:24
• I have added some clarification. I hope that helps. – Josh Feb 28 '18 at 8:33

Here is a solution for $N=1$ and $D=[a,b]$.
Note that in both cases, the kernel can be factorized as $K(x,y)=f(x)f(y)$. Thus the eigenvalue equation simplifies to $$\lambda \phi(x)=f(x)\int_a^b f(y)\phi(y)\,dy.$$ Hence, if $\lambda\neq 0$ (and this is the interesting case), then $\phi(x)=\alpha f(x)$ for some $\alpha\in \mathbb{R}$ and consequently $$\lambda=\int_a^b f(x)^2\,dx.$$ To get a normalized eigenfunction, we must have $$1=\int_a^b \alpha^2 f(x)^2\,dx,$$ that is, $\alpha=\left(\int_a^b f(x)^2\,dx\right)^{-1/2}$. It follows that $$\lambda \phi(x)\phi(y)=\left(\int_a^b f(x)^2\,dx\right)\left(\int_a^b f(x)^2\,dx\right)^{-1}f(x)f(y)=K(x,y),$$ as was expected from Mercer's theorem.
In the case of $K_1$ this gives the (only) non-zero eigenvalue $\lambda=\frac 1 3(b^3-a^3)$ with corresponding eigenfunction $\phi(x)=\lambda^{-1/2}x$, for $K_2$ we get $\lambda=\frac 1 5(b^5-a^5)$ and $\phi(x)=\lambda^{-1/2}x^2$.
In higher dimension, at least if $D=[a_1,b_1]\times\dots\times[a_N,b_N]$, one can still decompose $K$ as $K(x,y)=f_1(x)g_1(y)+\dots+f_n(x)g_n(y)$ and try a similar approach, but things get messier (I don't know if you get explicit results, haven't spelled it out).