Let $k_1,\ldots,k_p$ be positive definite kernels defined on $\mathcal{X}\times\mathcal{X}$, where $\mathcal{X}$ is a non-empty set. $k_i$ is the reproducing kernel of the Reproducing Kernel Hilbert Space (RKHS) $\mathcal{H}_i$, which is endowed with an inner product $\langle\cdot,\cdot\rangle_{\mathcal{H}_i}$, from which the norm $\lVert\cdot\rVert_{\mathcal{H}_i}$ is induced ($i=1,\ldots,p$).

Now let $k$ be a real-valued function defined on $\mathcal{X}\times\mathcal{X}$, with $k=\sum_{i=1}^{p}k_i$. It is straightforward to prove (isn't it?) that $k$ is positive definite, since positive-definiteness is preserved under summation. Apparently, there corresponds a RKHS, $\mathcal{H}$ to $k$, with inner product $\langle\cdot,\cdot\rangle_{\mathcal{H}}$ and norm $\lVert\cdot\rVert_{\mathcal{H}}$.

What is the relation between the norm of $\mathcal{H}$ and the norms of $\mathcal{H}_i$, $i=1,\ldots,p$? Do we need to require that the spaces $\mathcal{H}_i$ are pairwise-perpendicular so that it holds

$$ \lVert\cdot\rVert_{\mathcal{H}}^2 = \sum_{i=1}^{p}\lVert\cdot\rVert_{\mathcal{H}_i}^2, $$

and, if so, what that would mean for the reproducing kernels $k_i$?

Moreover: Let $f\in\mathcal{H}$, how could its norm, $\lVert f \rVert_{\mathcal{H}}$, be expressed in terms of the norms of the spaces $\mathcal{H}_i$?

The above questions may seem vague, or even incorrect in the way they are stated. Please feel free to suggest approaches and/or corrections (for instance, should the title be changed?), if necessary. Also, please feel free to discuss!

  • 1
    $\begingroup$ A good answer to that question is given by math.stackexchange.com/questions/1085200/…, in the case of $p=2$ but I think it can be fairly well generalized. $\endgroup$ – Zaccharie Ramzi Jan 22 '17 at 19:55
  • $\begingroup$ @ZaccharieRamzi, thank you very much for your comment! I haven't worked in this for a while, but I'll definitely return at some point. :) $\endgroup$ – nullgeppetto Jan 23 '17 at 7:54

If $k_1,\dots,k_p$ are kernels with RKHSs $\mathcal{H}_i$, then $k=\sum_i k_i$ is indeed a positive definite kernel again without further assumptions. The relation $\lVert f\rVert_{\mathcal{H}}^2 = \sum_{i=1}^{p}\lVert f\rVert_{\mathcal{H}_i}^2$ holds automatically for all functions that are in the intersection of all participating RKHSs. There is no additional condition of "pairwise-perpendicularity".

The interesting question is, which functions are in this intersection? Because of the RKHS construction and the dominance of norms one might guess $\mathcal{H}\subseteq \mathcal{H_i}$, so the intersection is $\mathcal{H}$ itself. But that would still require a proof.

  • $\begingroup$ Many thanks Mr. Lampert! This is an insightful, yet laconic answer. It still requires a proof, indeed, but now I have something to work with. By the way, if you are this person (I'm quite sure this is you), I am really impressed by you work! $\endgroup$ – nullgeppetto Jul 31 '14 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.