# Inner product from Fourier-like kernel

A kernel $K\colon [0,1]^s\times [0,1]^s \rightarrow\mathbb{R}$ is a symmetric and positive semi-definite function (meaning that for any $v_1,\ldots,v_m\in [0,1]^s$ and any $m\geq 1$, the matrix $[K(v_i,v_j)]_{i,j}$ is symmetric and positive semi-definite).

For any fixed $u\in [0,1]^s$, define the map $K_u = K(u,\cdot)\colon [0,1]^s\rightarrow\mathbb{R}$, and define the space of functions

$$\mathcal{H}_{K,0} = \{f = \sum_{i=1}^m a_iK_{v_i}\ |\ v_i\in [0,1]^s, a_i\in\mathbb{R}, m\geq 0\}$$

with the inner product

$$\langle f,g\rangle_K = \left\langle\sum_{i=1}^ma_iK_{v_i},\sum_{j=1}^l b_jK_{w_j}\right\rangle_K=\sum_{i=1}^m\sum_{j=1}^l a_ib_j K(v_i,w_j).$$

The completion $\mathcal{H}_K$ of the space $\mathcal{H}_{K,0}$ is called a reproducing kernel Hilbert space with kernel $K$.

I came across this example that I do not understand. First, I suppose you can extend the above to working over $\mathbb{C}$ in the natural way. Second, define the kernel $K$ by

$$K(u,v) = \sum_{h\in\mathbb{Z}^s} w(h)e^{2\pi i h^T(u-v)},$$

where $w(h)$ are non-negative weights, such that $\sum \limits_{h\in\mathbb{Z}^s} w(h)<\infty$.

Here is the part I don't understand: The corresponding inner product is $$\langle f,g\rangle_K = \sum_{h\in\mathbb{Z}^s} (w(h))^{-1} \hat{f}(h)\hat{g}^*(h),$$

where $\hat{f}(h)$ are the Fourier coefficients of $f$, and $\hat{g}^*(h)$ are the complex conjugate Fourier coefficients of $g$ (and when $w(h)=0$, then $1/w(h)=\infty$).

Why is this the corresponding inner product, and where do the Fourier coefficients come in? If

$$f(u) = \sum_{h\in\mathbb{Z}^s} \hat{f}(h)e^{2\pi i h\cdot u},$$ where $\hat{f}(h)$ are the Fourier coefficients of $f$, and similarly for $g$, then I don't see how $f$ and $g$ are elements of the space $\mathcal{H}$ in the first place, or why the inner product has the form it does.

It is sufficient to check things for $\mathcal{H}_{K,0}$. Let's look at the Fourier coefficients of the function $$f=\sum_{i=1}^{m}a_{i}K_{u_{i}}.$$ By definition $$K(u,v)=\sum_{h\in\mathbb{Z}^{s}}w(h)e^{2\pi ih^{T}(u-v)},$$ and so $$f(v)=\sum_{i=1}^{m}a_{i}\sum_{h\in\mathbb{Z}^{s}}w(h)e^{2\pi ih^{T}(u_{i}-v)}=\sum_{h\in\mathbb{Z}^{s}}\left(w(h)\sum_{i=1}^{m}a_{i}e^{2\pi ih^{T}u_{i}}\right)e^{-2\pi ih\cdot v},$$ which implies that $$\hat{f}(h)=w(-h)\sum_{i=1}^{m}a_{i}e^{-2\pi ih\cdot u_{i}}.$$ Similarly if $$g(v)=\sum_{j=1}^{l}b_{j}K_{w_{j}}(v)$$ then we have $$\hat{g}(h)=w(-h)\sum_{j=1}^{l}b_{j}e^{-2\pi ih\cdot w_{l}}.$$ Thus $$\sum_{h\in\mathbb{Z}^{s}}\frac{1}{w(h)}\hat{f}(h)\overline{\hat{g}(h)}=\sum_{h\in\mathbb{Z}^{s}}\frac{1}{w(-h)}\hat{f}(-h)\hat{g}(h)$$ $$=\sum_{h\in\mathbb{Z}^{s}}w(h)\sum_{j=1}^{l}b_{j}e^{-2\pi ih\cdot w_{l}}\sum_{i=1}^{m}a_{i}e^{2\pi ih\cdot u_{i}}=\sum_{i=1}^{m}\sum_{j=1}^{l} \sum_{h\in\mathbb{Z}^{s}}w(h) a_{i}b_{j}e^{2\pi ih^{T}(u_{i}-w_{j})},$$ and since this equals $$\sum_{i=1}^m\sum_{j=1}^l a_i b_j K(u_i,w_j)=\langle f,g\rangle,$$ the proof is finished.
• Thank you, Eric, this is probably the right way to think about this problem. I was looking at a general $f$ with corresponding Fourier expansion, and tried to fit this into the framework, but I suppose that not all Fourier expandable functions are elements of $\mathcal{H}_K$ (for some reason I cannot award you points for another 17 hours). – Mankind Nov 11 '15 at 18:27
• @HowDoIMath: There's also the problem of the definition of the inner product on $\mathcal{H}_K$. Since we defined this space as the completion of $\mathcal{H}_{K,0}$, the inner product is defined as a limit of inner products of functions in $\mathcal{H}_{K,0}$. – Eric Naslund Nov 11 '15 at 18:46