Let $K: X \times X \rightarrow \mathbb{C}$ be a positive definite kernel on a set $X$, i.e. for any $x_1, \cdots, x_n \in X$, the matrix

$$ [K(x_i, x_j)]_{ij} \in \mathbb{C}^{n \times n} $$

is positive definite.

The reproducing kernel Hilbert space $\mathcal{H}_K$ with kernel $K$ is the Hilbert space closure of the set $\{ K_x(y) = K(x,y)\}$ with the inner product

$$ \langle K_x(y), K_{x'}(y) \rangle_{\mathcal{H}_K} = K(x, x'). $$


Consider the vector space of functions on $X$ (no additional structure is assumed on $X$). Define an inner product by

$$ \langle \delta_x, \delta_{x'} \rangle = K(x, x'). $$

Then the resulting Hilbert space $\mathcal{H}$ is isomorphic to $\mathcal{H}_{K}$ via the isomorphism

$$ \delta_x(y) \mapsto K_x(y). $$

  1. $\mathcal{H}$ seems a much more natural and straight forward object than $\mathcal{H}_K$, so what is gained by the "reproducing kernel" construction? The reproducing kernel construction makes pointwise evaluation an element of the dual...why is this useful?

  2. In particular, what are some examples of specific contexts where $\mathcal{H}_K$ is more handy than $\mathcal{H}$ (I am guessing they must exist)?

  • 1
    $\begingroup$ Why does $\langle \delta_x, \delta_{x'} \rangle = K(x,x')$ define an inner product? The $\delta_x$ don't form a Hamel basis for $F(X,\mathbb{C})$ so there isn't just one extension from that to the whole of $F(X, \mathbb{C})$. $\endgroup$ Jan 12, 2022 at 23:10

1 Answer 1


A very practical application of this is to Machine Learning, specifically Support Vector Machines and general kernel methods. The problem setup is that you have a bunch of points $(\mathbf{x}_i,y_i)$, with $y_i$ being the value you want to predict from $\mathbf{x}_i$. So you start with some training set of points, and you want to construct a function that predicts $y$ given $\mathbf{x}$. If you want a linear predictor, then usual optimization theory works. But if you want a non-linear predictor, then things are much more difficult. Specifically, what happens is that it becomes extremely expensive to do operations as simple as multiplication and addition if your space $X$ is huge, say 50 dimensional. As well, with something like a Support Vector Machine, you are trying to effectively find a transformation $\phi$ of your set of points so that you can split different categories of points by hyperplanes. You can imagine how difficult such a problem would be directly.

The brilliance of kernel methods, specifically involving the so called kernel trick is that one can reduce these kinds of problems to calculating inner products of your data (which is relatively fast), and then working with the reproducing kernel, where one has access to Mercer's theorem.

  • $\begingroup$ So if $\mathcal{H}$ and $\mathcal{H}_K$ are isomorphic, why can the same not be done with just $\mathcal{H}$? $\endgroup$
    – Michael
    Nov 27, 2014 at 6:28
  • 1
    $\begingroup$ More precisely, what does that specific representation via reproducing kernel give you in that context? $\endgroup$
    – Michael
    Nov 28, 2014 at 7:27
  • $\begingroup$ The main use is to go from a infinite dimensional space $\mathcal{H}$ to a finite dimensional one. $\endgroup$
    – Næreen
    Feb 10, 2016 at 10:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .