Questions tagged [reproducing-kernel-hilbert-spaces]

Filter by
Sorted by
Tagged with
1 vote
0 answers
13 views

Inverse push-forward on RKHS

I am considering an infinite dimensional separable RKHS $H$ of functions from $E$ to $\mathbb{R}$, where $E$ is any measurable space. I denote by $\phi:E \rightarrow H$ the canonical feature map of $H$...
user avatar
  • 457
0 votes
1 answer
18 views

Differentiability of functions in reproducing kernel Hilbert space

Consider $\mathcal H_k$ to be the RKHS of a reproducing kernel $k$. I am interested in the differentiability properties of $\mathcal H_k$ as a space of functions. More precisely, is there a link ...
user avatar
  • 23
-1 votes
0 answers
5 views

Pros and cons of Nadaraya–Watson estimator vs. RKHS method?

Recently I've been reading some materials about nonparametric methods. Two methods related to the word "kernel" rasied my interest-- Nadaraya–Watson estimator and RKHS method. What's the ...
user avatar
0 votes
0 answers
9 views

Reproducing kernel Hilbert space norm as a smoothness functional

Let $K:X \times X \rightarrow \mathbb{R}$ be a Mercer kernel with an associated RKHS $H$ then the norm $|f|_H^2$ can be used as a way to ensure that $f$ is smooth in $H$. If i understand correctly, ...
user avatar
  • 165
2 votes
1 answer
52 views

Closure of balls in Reproducing Kernel Hilbert Space (RKHS)

Let $X \subset \mathbb{R}^m$ be compact, and $k: X\times X \rightarrow \mathbb{R}$ be a universal kernel function, in the sense that the corresponding RKHS $\mathcal{H}_k$ is dense in $C(X)$ under the ...
user avatar
  • 23
1 vote
1 answer
42 views

prove If $k_1$ and $k_2$ are positive semidefinite kernels then $min\{k_1, k_2\}$ and $max\{k_1, k_2\}$ are psd too.

I can prove for $R+$ the function $min(x,y)$ is a positive semidefinite kernel. But I'm stuck in proving the following statement. Suppose $k_1(x,y)$ and $k_2(x,y)$ are positive semidefinite kernels ...
user avatar
1 vote
0 answers
25 views

reproducing kernel hilbert space notation

I'm trying to understand reproducing kernel Hilbert spaces (RKHSs) from scientific papers, however I don't find any gentle introduction. However, my main problem, at the moment, seems to be to ...
user avatar
0 votes
0 answers
28 views

Moore-Aronszajn Theorem and Mercer theorem for the kernel trick

I have been reading about the RKHS and the kernel trick in Machine Learning mainly from https://ngilshie.github.io/jekyll/update/2018/02/01/RKHS.html (1) and https://arxiv.org/pdf/2106.08443.pdf (2). ...
user avatar
  • 165
1 vote
0 answers
46 views

Are there $\| \cdot \|_{\infty}$ Kernels?

Some kernels used in machine learning are linked to metrics via the negative exponential function $f(t) = e^{-t^p}$. The most prominent example is the Gaussian RBF kernel $$K(x,y) = e^{-\sigma^2 \|x-y\...
user avatar
1 vote
1 answer
79 views

Bochner's Theorem and Universal Kernels

Bochner's theorem asserts that a shift-invariant and properly scaled continuous kernel $K(x,y) = k(x-y)$ is positive definite (and hence a reproducing kernel of some RKHS) if and only if its Fourier ...
user avatar
0 votes
0 answers
38 views

Finding a Kernel Matrix on $ \mathbb{R}^n$

Let $ X = \mathbb{R}^n$ with inner product $⟨x, y⟩ = y^TQx$ for some symmetric positive definite matrix $Q$. I have a few questions about this setup and generally about the kernels. 1- A kernel is the ...
user avatar
  • 1,311
1 vote
0 answers
18 views

Universal approximation of neural networks

I am currently dealing with the topic of reproducing kernel Hilbert spaces (RKHS) given the draft book of Francis Bach. As a background knowledge for my current problem define: \begin{align} &H_1=\...
user avatar
  • 53
0 votes
0 answers
19 views

Reproducing kernel hilbert spaces: variation norm in arbitrary dimensions

I am currently dealing with the topic of reproducing kernel Hilbert spaces (RKHS) given the draft book of Francis Bach. As a background knowledge for my current problem define: \begin{align} &H_1=\...
user avatar
  • 53
0 votes
0 answers
25 views

Verify solution to linear Fredholm integral equation of the second kind

Let $\int_a^b C_X(t, s)\psi_k(s)ds = \lambda_k\psi_k(t)$, which corresponds to a homogeneous linear (Fredholm) integral equation of the second kind. Where $C_X(t, s)$ is the covariance function and is ...
user avatar
  • 1,311
2 votes
0 answers
24 views

Explicit calculation of the arccos-1-kernel

I have given the following problem from the draft book of Francis Bach: "For $(w,b/R)$ uniform on the sphere and for the ReLU activation, compute the associated kernel as a function of the cosine ...
user avatar
  • 53
-2 votes
1 answer
96 views

How to prove the min function is a kernel

How to prove that min(x,y) is a kernel. I got some hints online $$ min(x,y) = \int_0^\infty 1 [s \leq min(x,y)]ds = \int_0^\infty 1 [s \leq x] 1 [s \leq y]ds $$
user avatar
  • 1
2 votes
1 answer
48 views

Direct sum of reproducing kernel Hilbert spaces (RKHS)

I am currently diving into the theory of reproducing kernel Hilbert spaces and am just at the beginning of understanding the background of reproducing kernels. I have stumbled upon the following ...
user avatar
  • 53
2 votes
1 answer
64 views

The utility of kernel methods like RKHS in machine learning

In machine learning framework, kernel methods are widely used to find the close-form solution of a optimization problem, which restricts the solution in an RKHS. However, it really puzzles me that ...
user avatar
  • 21
2 votes
1 answer
48 views

Compactness of embeddings of Reproduing Kernel Hilbert Spaces with almost surely equal kernels

Let $\Omega \neq \emptyset$ and $\mu$ be a probability measure on $\Omega.$ Consider two reproducing kernels $k_1,k_2:\Omega \times \Omega \rightarrow \mathbb{R},$ such that they both represent the ...
user avatar
1 vote
1 answer
35 views

Penalized form of the nonparametric least-squares estimate

Consider the class of twice continuously differentiable functions $f: [0,1]\to R$ and for a given squared radius $R>0$, define the function class $$ \mathcal{F}(R):=\left\{f:[0,1]\to R: \int_0^1(f''...
user avatar
  • 1,602
2 votes
0 answers
36 views

Bochner integral on function spaces

Let $(X,\Sigma,\mu)$ be a measure space and $B$ be a Banach space. A Bochner-measurable function $f: X \rightarrow B$ is Bochner integrable if and only if $$ \int_{X}\|f(x)\|_{B} d \mu(x)<\infty $$ ...
user avatar
  • 457
0 votes
0 answers
55 views

Reproducing property of Szegő kernel

I have an embarrassingly basic question about the Szegő kernel on the Hardy space $H^2$ over the right half plane. It seems I have forgotten as much complex analysis as I ever knew... A kernel $k(s, ...
user avatar
  • 312
2 votes
2 answers
229 views

Why is this functional derivative equal to $0$?

I am currently reading the paper Exponential Convergence Rates in Classification (2005) by Vladimir Koltchinskii and Oleksandra Beznosova, and I'm having trouble following the proof of the main result,...
user avatar
  • 1,953
0 votes
1 answer
50 views

RKHS of discontinous function

I have a discountinous function $f$ which I would like show it as $f \in \mathcal{H}_k(\mathcal{X})$ where $\mathcal{H}_k(\mathcal{X})$ is a RKHS generated by kernel $k$ in domain $\mathcal{X}$. Is it ...
user avatar
0 votes
1 answer
36 views

Prove that RKHS of translation invriant kernel is a Hilbert space

I am reading a lecture on RKHS of kernels and here is an exercise I have not been able to solve Given a translation invariant kernel $$ k(x,y) = \varphi (x-y) $$ The theorem states that once $\varphi$ ...
user avatar
2 votes
2 answers
219 views

Why is $L^2$ not a RKHS?

Let's take $L^2[0,1]$ as an example. Obviously $L^2[0,1]$ is not a Hilbert space because you can have a function $f(0)=1$ but equal to $0$ everywhere else, so you don't have an unique element with ...
user avatar
  • 59
2 votes
0 answers
44 views

RKHS for one-dimensional discontinuous jump functions

I would like to do Kernel Regression on the space of functions from $\mathbb{R}$ to $\mathbb{R}$ which have a countable number of jump discontinuities, and are otherwise continuous (in particular at ...
user avatar
  • 135
3 votes
1 answer
107 views

Convergence theorems for Kernel SVM and Kernel Perceptron

Context Some time ago I asked whether SVMs could work on arbitrary Hilbert spaces, my motivation for asking it was due to my discomfort towards the kernelized version of SVM, which, in my mind, ...
user avatar
2 votes
1 answer
103 views

The dot product is a valid kernel. Is the integral too?

One of the most well known kernels (which in my case is used as a covariance matrix of a Gaussian process) is the dot product kernel $k\left(x, x'\right)= x \cdot x' =\sum_{i=1}^{n} x_{i} x'_{i}$ and $...
user avatar
  • 301
1 vote
1 answer
57 views

Is the span of a universal kernel on a compact metric space dense in the space of continuous functions?

Suppose $X$ is a compact metric space, and $k:X\times X\to\mathbb{R}$ is a continuous, universal kernel on $X$. By definition of a universal kernel, the RKHS corresponding to $k$, defined as $$ \...
user avatar
  • 11
0 votes
0 answers
44 views

Does RKHS norm preserve inequality in L2?

Suppose $\mathcal{H}$ is a reproducing kernel Hilbert space with reproducing kernel $K$. Assume that $f,g$ are two elements in $L_{2}$ and also in $\mathcal{H}$. My question is what kind of condition ...
user avatar
  • 3
2 votes
1 answer
98 views

Orthogonalisation in Reproducing Kernel Hilbert Space (RKHS) and null space

Let $\mathcal{X}$ be a set equipped with a positive definite kernel $K$ with value $K(x,\, x')$. Let $\mathcal{K}$ be the corresponding RKHS and consider a closed linear subspace $\mathcal{F}$ in $\...
user avatar
  • 111
1 vote
0 answers
49 views

Is a kernel just a symmetric, positive semi-definite, and continuous matrix?

From page 9 of these course notes, A function $k : \mathcal{X} \times \mathcal{X} \mapsto \mathbb{R}$ is a kernel if $k$ is symmetric: $k(x,y) = k(y,x)$ $k$ gives rise to a positive semi-definite &...
user avatar
  • 809
0 votes
1 answer
44 views

Sum of sample expectations of function of 2 variables (Understanding MMD)

I'm trying to understand the proofs to bound Maximum Mean Discrepancy (MMD) in the paper "A Kernel Two-Sample Test" by Gretton et al. (2012). These are given in the appendix. Specifically, I ...
user avatar
  • 195
3 votes
0 answers
39 views

When are signed measures isomorphic to a RKHS?

Let $\mathcal{H}$ be a reproducing kernel Hilbert space of functions over $X$, with a bounded kernel $\mathcal{K}: X\times X\to \mathbb{R}$. Let us assume there is a sigma-algebra over $X$ such that $\...
user avatar
0 votes
0 answers
32 views

Reference Request: Generalization bounds for kernel ridge regression

Let $A$ be an $n\times m$ matrix, $(X_n,Y_n)_{n=1}^N$ be a set of i.i.d. random vectors taking values in $\mathbb{R}^m\times \mathbb{R}^k$ and define the $M$-estimator $S^{(N)}$ by: $$ S^{(N)}(x)\...
user avatar
0 votes
0 answers
19 views

What function yields a projection matrix with the smallest Frobenius norm to a given projection matrix?

Phrasing Attempt 1 If I have one function $f_1: \mathcal{X} \rightarrow \mathbb{R}^{D_1}$ that yields a particular projection matrix $P_1 \in \mathbb{R}^{N \times N}$, how do I find the function $f_2: ...
user avatar
0 votes
1 answer
60 views

the distance between two kernels

I wonder if there is some existing work discussing measuring the similarity between two kernels, is there any distance defined between two kernels? A more specific question related to that is, ...
user avatar
  • 445
0 votes
0 answers
32 views

Prove a function is conditionally positive semidefinite

This is from an "exercise for the reader" in my professor's slides that I really can't wrap my head around. We know from Boughorbel et al., 2005 that $k(\mathbf{x}, \mathbf{x'}) = -\|\mathbf{...
user avatar
  • 1
0 votes
0 answers
22 views

pre-image of kernel Hilbert space

Suppose the Hilbert Space $H$ contains all functions $k_w: x\to k(w,x)$, it's the fact that every $w\in R^n$ would have a corresponding in $k_w\in H$, but I'm wondering that is every function in $H$ ...
user avatar
0 votes
0 answers
30 views

nonlinear features mapping to the RKHS

Edited: We usually use linear features mapping in RKHS. But how to handle a nonlinear features mapping? For example: if we are given a non linear mapping $$\phi:R^d\to R^{d+d(d-1)/2}: x = \begin{...
user avatar
2 votes
1 answer
111 views

Can RKHS of Gaussian kernels over $\mathbb{R}^d$ have a non-zero element which is zero on a linear subspace $R^k\subset R^d$ where $k>0$?

I have been thinking on this problem for at least a day now, and I thought the answer is a resounding no. It seems I am wrong, but wanted to double-check. I came across this: And from my understanding ...
user avatar
  • 1,301
0 votes
0 answers
15 views

Efficient computing of multiple kernel ridge regressions with same data (i.e. same kernel matrix, different regularization $\lambda$)

As is standard in Kernel Ridge Regression, let $K \in \mathbb{R}^{n \times n}$ be the kernel matrix corresponding to some data, and let $Y \in \mathbb{R}^n$ be the vector of predictions, and $\lambda \...
user avatar
  • 135
0 votes
0 answers
20 views

finding kernel preserving transformations

My instructor started talking about kernels and defined a kernel on a set $X$ to be a function $K:X \times X \rightarrow \mathbb{R}$ that is symmetric and positive (semi) definite in the sense that ...
user avatar
1 vote
1 answer
56 views

What is the dimensionality of $\phi(x)$ in terms of $n$ in kernel function?

Kernel Function: $K(x,y) = (b + x^Ty)^2$ is a quadratic polynomial kernel, where $x, y ∈ R^n $and $b > 0$. Provide a feature mapping $\phi(x)$ such that $K(x,y) = \phi(x)^T\phi(y)$. What is the ...
user avatar
  • 11
0 votes
0 answers
25 views

"Learning with Kernels" Theorem 2.10, Mercer's theorem

I am have trouble proving one of the theorems listed in the book Learning with Kernels: In particular, it is assumed that $\mu(X) < \infty$, I know that $T_k$ is a compact self-adjoint operator on ...
user avatar
0 votes
0 answers
34 views

Prove the uniqueness of this bounded linear operator

I'm reading the paper Charles R. Baker. Joint measures and cross-covariance operators. Transactions of the American Mathematical Society, 186:273–289, 1973. I'm stuck in the last part of the proof of ...
user avatar
  • 61
0 votes
0 answers
54 views

Inversion Formula Gaussian Convolution

I am looking at the following 2004 paper by S. Saitoh, called "Approximate real inversion formulas of the Gaussian convolution": https://www.researchgate.net/publication/...
user avatar
1 vote
1 answer
31 views

Prove that $f$ is a linear combination of the functions $k_{y_i}(x) = min\{x,y_i\}$.

This is exercise 2.4 of "An Introduction to the Theory of Reproducing Kernel Hilbert Spaces" by Paulsen, and it states: Let $y_0 = 0 < y_1 < \dots < y_n$ be given and let $f: [0,\...
user avatar
  • 195
1 vote
1 answer
34 views

For given Pontryagin space the norm induced topology is not depending on the fundamental decomposition

For given Hilbert space $H$ consider its corresponding antispace, i.e. the vector space endowed with its negative inner product. $\Pi$ is now called a Pontryagin space if it can be written as a direct ...
user avatar