There must be something very basic that I am missing.

If I understood correctly from wikipedia, positive-definiteness of kernel $K(\cdot,\cdot)$ is sufficient for universality of this kernel. Then, according to same wiki page, corresponding Reproducing Kernel Hilbert Space (RKHS) $\mathcal H_K$ must be dense in $\mathcal C(\mathbb R)$. More specifically, for each kernel $K$ we can build corresponding RKHS as following:

$$\mathcal H_K = \text{span}\{K(x, .) : x \in \mathbb R\}$$ with scalar product given by

$$ f(x) = \sum_i^n \alpha_i K(x_i, x)$$ $$ g(x) = \sum_j^m \beta_j K(x_j, x)$$

$$ f \cdot g = \sum_i^n \sum_j^m \beta_j \alpha_i K(x_j, x) \cdot K(x_i, x) = \sum_i^n \sum_j^m \beta_j \alpha_i K(x_j, x_i) $$

However, even though $K(x, y) = x^T y$ is positive definite, it seem to consist only of linear functions, so its closure is far from being $\mathcal C(\mathbb R)$.

  • $\begingroup$ Could you recall what you mean by abbreviation RKHS ? $\endgroup$
    – Jean Marie
    Feb 8 '17 at 22:55
  • $\begingroup$ @JeanMarie updated. $\endgroup$
    – Ben Usman
    Feb 8 '17 at 23:26
  • $\begingroup$ Something wrong with my question? $\endgroup$
    – Ben Usman
    Feb 10 '17 at 16:40
  • 1
    $\begingroup$ Have you seen this document (math.uh.edu/~vern/rkhs.pdf) ? $\endgroup$
    – Jean Marie
    Feb 10 '17 at 17:09

Now I have figured out, thank you, @JeanMarie.

Several mistakes I made:

$$ \mathcal H = \overline{\text{span}\{K(x, \cdot), x \in X\}}$$, so $\mathcal H$ has infinite sums.

Second mistake and answer to my question [1, page 139]:

If kernel is shift-invariant $h(x-y)=K(x, y)$ and we represent it in Fourier domain as $$h(t) = \int e^{-i\langle t, \omega \rangle} \mu(d\omega)$$ then if $\text{supp}\{\mu\}$ is entire space, then $K$ is universal, so $H_K$ is dense in the space of all bounded continuous functions.

  • $\begingroup$ correct me if I'm wrong $\endgroup$
    – Ben Usman
    Feb 12 '17 at 1:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.