0
$\begingroup$

Premise:

A function $K: \mathbb R^d \times \mathbb R^d \to \mathbb R$ is called a kernel function on $\mathbb{R}^d$ if there exists a Hilbert space $\mathcal{H}$ and a map $\phi: \mathbb R^d \to \mathcal{H}$ such that for any $\mathbf x, \mathbf y\in \mathbb{R}^d$: \begin{equation} \label{eq:kerdef} K(\mathbf x,\mathbf y) = \langle \phi(\mathbf x), \phi(\mathbf y) \rangle_\mathcal{H}, \end{equation} where $ \langle\cdot,\cdot\rangle$ is an inner product.

I have recently noticed that under this definition $ \langle\cdot,\cdot\rangle$ is not necessary the standard inner product (dot product).

I have thought of the following example. Consider the degree-two polynomial kernel on $\mathbb R^2$: \begin{equation} K(\mathbf x,\mathbf y) = (\mathbf x\cdot \mathbf y)^2. \end{equation} Then, the following is a valid feature map for this kernel: \begin{equation} \phi(\mathbf x) = (2\mathbf x_1 \mathbf x_1,~ 2\mathbf x_1 \mathbf x_2,~ 2\mathbf x_2 \mathbf x_1,~ 2\mathbf x_2 \mathbf x_2 ), \end{equation} considering the feature space $\mathcal H=\mathbb R^4$, with the custom inner product: \begin{equation} \langle\mathbf x, \mathbf y \rangle = \frac{\mathbf x\cdot \mathbf y}{4}. \end{equation}

Proof: \begin{align} \langle \phi(\mathbf x), \phi(\mathbf y) \rangle_\mathcal{H} &= (2\mathbf x_1 \mathbf x_1,~ 2\mathbf x_1 \mathbf x_2,~ 2\mathbf x_2 \mathbf x_1,~ 2\mathbf x_2 \mathbf x_2 ) \cdot (2\mathbf y_1 \mathbf y_1,~ 2\mathbf y_1 \mathbf y_2,~ 2\mathbf y_2 \mathbf y_1,~ 2\mathbf y_2 \mathbf y_2 )/4\\ &= (\mathbf x \cdot \mathbf y)^2 \\&= K(\mathbf x, \mathbf y) \end{align}

Question:

Can you provide a less trivial example of kernel function, feature map and feature space, with an inner product that is not the dot product?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.