4
$\begingroup$

Let $\mathcal{X}$ be a set equipped with a positive definite kernel $K$ with value $K(x,\, x')$. Let $\mathcal{K}$ be the corresponding RKHS and consider a closed linear subspace $\mathcal{F}$ in $\mathcal{K}$ and $\mathcal{G}$ be the orthogonal supplement of $\mathcal{F}$ in $\mathcal{K}$. Beside the the inner product $\langle\,, \,\rangle_{\mathcal{K}}$ we can define a new inner product $\langle\,, \,\rangle_{\widetilde{\mathcal{K}}}$ on $\mathcal{K}$ by

$$ \left\langle h, \, h' \right\rangle_{\widetilde{\mathcal{K}}} := \left\langle P.h, \, P.h' \right\rangle_{\mathcal{K}} $$

where $P.h$ denotes the orthogonal projection on $\mathcal{G}$. Then $\mathcal{H}$ equipped with $\langle\,, \,\rangle_{\widetilde{\mathcal{K}}}$ becomes a new RKHS with null space $\mathcal{F}$. How can we relate the semi-definite positive kernel $\widetilde{K}$ on $\mathcal{X}$ to the original kernel $K$?

Comments. In the RKHS framework for splines the original kernel is often taken as $K(x, \, x') = F(x,\,x') + G(x,\,x')$ where $F$ and $G$ are semi-positive definite kernels with RKHS $\mathcal{F}$ and $\mathcal{G}$ which form a direct sum of $\mathcal{K}$ hence we simply have $\widetilde{K}(x, \, x') = G(x, \, x')$ in this case. The null space $\mathcal{F}$ is then usually finite-dimensional. Rather than starting from a kernel which is a direct sum, I would like to start from a general kernel hence to achieve some kind of subtraction of kernels. Of special interest is the case where $\mathcal{F}$ is finite-dimensional and an example with a closed form for $K(x, \, x')$ and $\widetilde{K}(x, \, x')$ would be great. The question relates to universal Kriging applications where $\mathcal{X}$ is a domain in $\mathbb{R}^d$, $\mathcal{F}$ is spanned by so-called trend functions and $K$ is a kernel given in closed form: Whittle-Matérn, square-exponential, ...

$\endgroup$
2
  • $\begingroup$ I have trouble to understand your question "can we relate the kernels". You state the relation between $K$ and $\tilde K$ correctly and subtraction is also straigthforward under your assumptions (the two subspaces being closed and orthogonal). So, what else do you want? $\endgroup$
    – g g
    Dec 17, 2021 at 10:53
  • $\begingroup$ Using for instance the exponential kernel $K(x, \,x') = \exp\{-\kappa |x -x'|\}$ on a real interval $[0, \, T]$, the RKHS and its norm are known. If I take $\mathcal{F}$ as generated by the constant function $1$, what is the expression of $\widetilde{K}(x,\,x')$ and what are the steps in its derivation? I found $\widetilde{K} = K - 2 / (2 + \kappa T)$ which seems strange and I think that my derivation was wrong. Also I believe that $\widetilde{K}$ can not be positive definite. $\endgroup$
    – Yves
    Dec 17, 2021 at 14:38

1 Answer 1

2
$\begingroup$

The identity map of the Hilbert Space $\mathscr{H}$ can be written as $I=P + (I-P)$ where $P$ and $I-P$ are the orthogonal projections on your spaces $\mathscr{F}$ and $\mathscr{G}$. Now set $k_x(\cdot)=K(x,\cdot)$ then due to orthogonality
$$ K(x,y) = \langle k_x,k_y\rangle = \langle Pk_x,Pk_y\rangle + \langle (I-P)k_x, (I-P)k_y\rangle.$$ and nothing holds you back from simply writing $$ \tilde K(x,y) = K(x,y) - \langle(I-P)k_x,(I-P)k_y\rangle.$$ Note that the RHS is always non-negative since projections reduce norms, i.e. $\lVert (I-P)k_x\rVert\leq \lVert k_x\rVert.$

Now to your concrete example. The orthogonal projection of the function $k_x$ on the complement of the space spanned by the constant function $1$ is: $$ \tilde{k}_x = k_x - \frac{\langle k_x,1\rangle}{\lVert 1 \rVert^2}1$$ and $$ \tilde K(x,y) = \langle \tilde{k}_x, \tilde{k}_y\rangle = \langle k_x - \frac{\langle k_x,1\rangle}{\lVert 1 \rVert^2}1, k_y - \frac{\langle k_y,1\rangle}{\lVert 1 \rVert^2}1\rangle = \langle {k}_x, {k}_y\rangle - \frac{\langle {k}_x, 1\rangle \langle {k}_y, 1\rangle}{\lVert 1\rVert^2}.$$

Assume $\tau=1$ for simplicity and note that the inner product of your space is $$ \langle f,g \rangle = f(0)g(0) + f(T)g(T) + \int_0^T f(t)g(t)\,dt + \int_0^T f'(t)g'(t)\,dt $$ which means the two inner products can be evaluated using $$ \langle k_x,1 \rangle = k_x(0) + k_x(T) + \int_0^T k_x(t)\,dt.$$

By using Gram-Schmidt the above works for projections on any finite dimensional space.

$\endgroup$
2
  • $\begingroup$ I am quite uncomfortable with the notations in the first two equations since $Px$ is only clear for me when $x$ is a function in the RKHS. I can not figure out what the norm $\|x \|$ is for $x \in \mathcal{X}$ which is not even a linear space. I think the right formalism is provided by Theorem 11 in the book by Berlinet and Thomas-Agnan. Also I realized that we must restrict to $\mathcal{G}$ to define a RKHS otherwise we get as semi-Hilbert space, not a Hibert space. $\endgroup$
    – Yves
    Dec 20, 2021 at 12:43
  • $\begingroup$ Of course, you are right and I corrected the notation. With respect to the restrictions: Since you assumed that $\mathscr{F}$ is closed, which it is in particular if it is finite dimensional, you do not have to put any additional requirements on $\mathscr{G}.$ This is no surprise in the finite dimensional case, since you can write down its Kernel explicitly. $\endgroup$
    – g g
    Dec 20, 2021 at 13:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .