Questions tagged [reproducing-kernel-hilbert-spaces]
The reproducing-kernel-hilbert-spaces tag has no usage guidance.
168
questions
1
vote
0
answers
13
views
Inverse push-forward on RKHS
I am considering an infinite dimensional separable RKHS $H$ of functions from $E$ to $\mathbb{R}$, where $E$ is any measurable space. I denote by $\phi:E \rightarrow H$ the canonical feature map of $H$...
- 457
0
votes
1
answer
18
views
Differentiability of functions in reproducing kernel Hilbert space
Consider $\mathcal H_k$ to be the RKHS of a reproducing kernel $k$. I am interested in the differentiability properties of $\mathcal H_k$ as a space of functions.
More precisely, is there a link ...
- 23
-1
votes
0
answers
5
views
Pros and cons of Nadaraya–Watson estimator vs. RKHS method?
Recently I've been reading some materials about nonparametric methods. Two methods related to the word "kernel" rasied my interest-- Nadaraya–Watson estimator and RKHS method.
What's the ...
- 1
0
votes
0
answers
9
views
Reproducing kernel Hilbert space norm as a smoothness functional
Let $K:X \times X \rightarrow \mathbb{R}$ be a Mercer kernel with an associated RKHS $H$ then the norm $|f|_H^2$ can be used as a way to ensure that $f$ is smooth in $H$. If i understand correctly, ...
- 165
2
votes
1
answer
52
views
Closure of balls in Reproducing Kernel Hilbert Space (RKHS)
Let $X \subset \mathbb{R}^m$ be compact, and $k: X\times X \rightarrow \mathbb{R}$ be a universal kernel function, in the sense that the corresponding RKHS $\mathcal{H}_k$ is dense in $C(X)$ under the ...
- 23
1
vote
1
answer
42
views
prove If $k_1$ and $k_2$ are positive semidefinite kernels then $min\{k_1, k_2\}$ and $max\{k_1, k_2\}$ are psd too.
I can prove for $R+$ the function $min(x,y)$ is a positive semidefinite kernel. But I'm stuck in proving the following statement.
Suppose $k_1(x,y)$ and $k_2(x,y)$ are positive semidefinite kernels
...
1
vote
0
answers
25
views
reproducing kernel hilbert space notation
I'm trying to understand reproducing kernel Hilbert spaces (RKHSs) from scientific papers, however I don't find any gentle introduction. However, my main problem, at the moment, seems to be to ...
- 209
0
votes
0
answers
28
views
Moore-Aronszajn Theorem and Mercer theorem for the kernel trick
I have been reading about the RKHS and the kernel trick in Machine Learning mainly from https://ngilshie.github.io/jekyll/update/2018/02/01/RKHS.html (1) and https://arxiv.org/pdf/2106.08443.pdf (2). ...
- 165
1
vote
0
answers
46
views
Are there $\| \cdot \|_{\infty}$ Kernels?
Some kernels used in machine learning are linked to metrics via the negative exponential function $f(t) = e^{-t^p}$. The most prominent example is the Gaussian RBF kernel
$$K(x,y) = e^{-\sigma^2 \|x-y\...
- 1,856
1
vote
1
answer
79
views
Bochner's Theorem and Universal Kernels
Bochner's theorem asserts that a shift-invariant and properly scaled continuous kernel $K(x,y) = k(x-y)$ is positive definite (and hence a reproducing kernel of some RKHS) if and only if its Fourier ...
- 1,856
0
votes
0
answers
38
views
Finding a Kernel Matrix on $ \mathbb{R}^n$
Let $ X = \mathbb{R}^n$ with inner product $⟨x, y⟩ = y^TQx$ for some symmetric positive definite matrix $Q$. I have a few questions about this setup and generally about the kernels.
1- A kernel is the ...
- 1,311
1
vote
0
answers
18
views
Universal approximation of neural networks
I am currently dealing with the topic of reproducing kernel Hilbert spaces (RKHS) given the draft book of Francis Bach.
As a background knowledge for my current problem define:
\begin{align}
&H_1=\...
- 53
0
votes
0
answers
19
views
Reproducing kernel hilbert spaces: variation norm in arbitrary dimensions
I am currently dealing with the topic of reproducing kernel Hilbert spaces (RKHS) given the draft book of Francis Bach.
As a background knowledge for my current problem define:
\begin{align}
&H_1=\...
- 53
0
votes
0
answers
25
views
Verify solution to linear Fredholm integral equation of the second kind
Let $\int_a^b C_X(t, s)\psi_k(s)ds = \lambda_k\psi_k(t)$, which corresponds to a homogeneous linear (Fredholm) integral equation of the second kind. Where $C_X(t, s)$ is the covariance
function and is ...
- 1,311
2
votes
0
answers
24
views
Explicit calculation of the arccos-1-kernel
I have given the following problem from the draft book of Francis Bach:
"For $(w,b/R)$ uniform on the sphere and for the ReLU activation, compute the associated kernel as a function of the cosine ...
- 53
-2
votes
1
answer
96
views
How to prove the min function is a kernel
How to prove that min(x,y) is a kernel. I got some hints online
$$ min(x,y) = \int_0^\infty 1 [s \leq min(x,y)]ds = \int_0^\infty 1 [s \leq x] 1 [s \leq y]ds $$
- 1
2
votes
1
answer
48
views
Direct sum of reproducing kernel Hilbert spaces (RKHS)
I am currently diving into the theory of reproducing kernel Hilbert spaces and am just at the beginning of understanding the background of reproducing kernels. I have stumbled upon the following ...
- 53
2
votes
1
answer
64
views
The utility of kernel methods like RKHS in machine learning
In machine learning framework, kernel methods are widely used to find the close-form solution of a optimization problem, which restricts the solution in an RKHS. However, it really puzzles me that ...
- 21
2
votes
1
answer
48
views
Compactness of embeddings of Reproduing Kernel Hilbert Spaces with almost surely equal kernels
Let $\Omega \neq \emptyset$ and $\mu$ be a probability measure on $\Omega.$ Consider two reproducing kernels $k_1,k_2:\Omega \times \Omega \rightarrow \mathbb{R},$ such that they both represent the ...
- 68
1
vote
1
answer
35
views
Penalized form of the nonparametric least-squares estimate
Consider the class of twice continuously differentiable functions $f: [0,1]\to R$ and for a given squared radius $R>0$, define the function class
$$
\mathcal{F}(R):=\left\{f:[0,1]\to R: \int_0^1(f''...
- 1,602
2
votes
0
answers
36
views
Bochner integral on function spaces
Let $(X,\Sigma,\mu)$ be a measure space and $B$ be a Banach space. A Bochner-measurable function $f: X \rightarrow B$ is Bochner integrable if and only if
$$
\int_{X}\|f(x)\|_{B} d \mu(x)<\infty
$$
...
- 457
0
votes
0
answers
55
views
Reproducing property of Szegő kernel
I have an embarrassingly basic question about the Szegő kernel on the Hardy space $H^2$ over the right half plane. It seems I have forgotten as much complex analysis as I ever knew...
A kernel $k(s, ...
- 312
2
votes
2
answers
229
views
Why is this functional derivative equal to $0$?
I am currently reading the paper Exponential Convergence Rates in Classification (2005) by Vladimir Koltchinskii and Oleksandra Beznosova, and I'm having trouble following the proof of the main result,...
- 1,953
0
votes
1
answer
50
views
RKHS of discontinous function
I have a discountinous function $f$ which I would like show it as $f \in \mathcal{H}_k(\mathcal{X})$ where $\mathcal{H}_k(\mathcal{X})$ is a RKHS generated by kernel $k$ in domain $\mathcal{X}$. Is it ...
0
votes
1
answer
36
views
Prove that RKHS of translation invriant kernel is a Hilbert space
I am reading a lecture on RKHS of kernels and here is an exercise I have not been able to solve
Given a translation invariant kernel
$$
k(x,y) = \varphi (x-y)
$$
The theorem states that once $\varphi$ ...
- 394
2
votes
2
answers
219
views
Why is $L^2$ not a RKHS?
Let's take $L^2[0,1]$ as an example. Obviously $L^2[0,1]$ is not a Hilbert space because you can have a function $f(0)=1$ but equal to $0$ everywhere else, so you don't have an unique element with ...
- 59
2
votes
0
answers
44
views
RKHS for one-dimensional discontinuous jump functions
I would like to do Kernel Regression on the space of functions from $\mathbb{R}$ to $\mathbb{R}$ which have a countable number of jump discontinuities, and are otherwise continuous (in particular at ...
- 135
3
votes
1
answer
107
views
Convergence theorems for Kernel SVM and Kernel Perceptron
Context
Some time ago I asked whether SVMs could work on arbitrary Hilbert spaces, my motivation for asking it was due to my discomfort towards the kernelized version of SVM, which, in my mind, ...
- 175
2
votes
1
answer
103
views
The dot product is a valid kernel. Is the integral too?
One of the most well known kernels (which in my case is used as a covariance matrix of a Gaussian process) is the dot product kernel $k\left(x, x'\right)= x \cdot x' =\sum_{i=1}^{n} x_{i} x'_{i}$ and $...
- 301
1
vote
1
answer
57
views
Is the span of a universal kernel on a compact metric space dense in the space of continuous functions?
Suppose $X$ is a compact metric space, and $k:X\times X\to\mathbb{R}$ is a continuous, universal kernel on $X$. By definition of a universal kernel, the RKHS corresponding to $k$, defined as
$$ \...
- 11
0
votes
0
answers
44
views
Does RKHS norm preserve inequality in L2?
Suppose $\mathcal{H}$ is a reproducing kernel Hilbert space with reproducing kernel $K$. Assume that $f,g$ are two elements in $L_{2}$ and also in $\mathcal{H}$.
My question is what kind of condition ...
- 3
2
votes
1
answer
98
views
Orthogonalisation in Reproducing Kernel Hilbert Space (RKHS) and null space
Let $\mathcal{X}$ be a set equipped with a positive definite kernel
$K$ with value $K(x,\, x')$. Let $\mathcal{K}$ be the corresponding
RKHS and consider a closed linear subspace $\mathcal{F}$ in
$\...
- 111
1
vote
0
answers
49
views
Is a kernel just a symmetric, positive semi-definite, and continuous matrix?
From page 9 of these course notes,
A function $k : \mathcal{X} \times \mathcal{X} \mapsto \mathbb{R}$ is a kernel if
$k$ is symmetric: $k(x,y) = k(y,x)$
$k$ gives rise to a positive semi-definite &...
- 809
0
votes
1
answer
44
views
Sum of sample expectations of function of 2 variables (Understanding MMD)
I'm trying to understand the proofs to bound Maximum Mean Discrepancy (MMD) in the paper "A Kernel Two-Sample Test" by Gretton et al. (2012). These are given in the appendix. Specifically, I ...
- 195
3
votes
0
answers
39
views
When are signed measures isomorphic to a RKHS?
Let $\mathcal{H}$ be a reproducing kernel Hilbert space of functions over $X$, with a bounded kernel $\mathcal{K}: X\times X\to \mathbb{R}$. Let us assume there is a sigma-algebra over $X$ such that $\...
- 129
0
votes
0
answers
32
views
Reference Request: Generalization bounds for kernel ridge regression
Let $A$ be an $n\times m$ matrix, $(X_n,Y_n)_{n=1}^N$ be a set of i.i.d. random vectors taking values in $\mathbb{R}^m\times \mathbb{R}^k$ and define the $M$-estimator $S^{(N)}$ by:
$$
S^{(N)}(x)\...
- 133
0
votes
0
answers
19
views
What function yields a projection matrix with the smallest Frobenius norm to a given projection matrix?
Phrasing Attempt 1
If I have one function $f_1: \mathcal{X} \rightarrow \mathbb{R}^{D_1}$ that yields a particular projection matrix $P_1 \in \mathbb{R}^{N \times N}$, how do I find the function $f_2: ...
- 399
0
votes
1
answer
60
views
the distance between two kernels
I wonder if there is some existing work discussing measuring the similarity between two kernels, is there any distance defined between two kernels?
A more specific question related to that is, ...
- 445
0
votes
0
answers
32
views
Prove a function is conditionally positive semidefinite
This is from an "exercise for the reader" in my professor's slides that I really can't wrap my head around.
We know from Boughorbel et al., 2005 that $k(\mathbf{x}, \mathbf{x'}) = -\|\mathbf{...
- 1
0
votes
0
answers
22
views
pre-image of kernel Hilbert space
Suppose the Hilbert Space $H$ contains all functions $k_w: x\to k(w,x)$, it's the fact that every $w\in R^n$ would have a corresponding in $k_w\in H$, but I'm wondering that is every function in $H$ ...
- 31
0
votes
0
answers
30
views
nonlinear features mapping to the RKHS
Edited: We usually use linear features mapping in RKHS. But how to handle a nonlinear features mapping? For example: if we are given a non linear mapping $$\phi:R^d\to R^{d+d(d-1)/2}: x = \begin{...
- 31
2
votes
1
answer
111
views
Can RKHS of Gaussian kernels over $\mathbb{R}^d$ have a non-zero element which is zero on a linear subspace $R^k\subset R^d$ where $k>0$?
I have been thinking on this problem for at least a day now, and I thought the answer is a resounding no. It seems I am wrong, but wanted to double-check. I came across this:
And from my understanding ...
- 1,301
0
votes
0
answers
15
views
Efficient computing of multiple kernel ridge regressions with same data (i.e. same kernel matrix, different regularization $\lambda$)
As is standard in Kernel Ridge Regression, let $K \in \mathbb{R}^{n \times n}$ be the kernel matrix corresponding to some data, and let $Y \in \mathbb{R}^n$ be the vector of predictions, and $\lambda \...
- 135
0
votes
0
answers
20
views
finding kernel preserving transformations
My instructor started talking about kernels and defined a kernel on a set $X$ to be a function $K:X \times X \rightarrow \mathbb{R}$ that is symmetric and positive (semi) definite in the sense that ...
- 395
1
vote
1
answer
56
views
What is the dimensionality of $\phi(x)$ in terms of $n$ in kernel function?
Kernel Function: $K(x,y) = (b + x^Ty)^2$ is a quadratic polynomial kernel, where $x, y ∈ R^n $and $b > 0$. Provide a feature mapping $\phi(x)$ such that $K(x,y) = \phi(x)^T\phi(y)$. What is the ...
- 11
0
votes
0
answers
25
views
"Learning with Kernels" Theorem 2.10, Mercer's theorem
I am have trouble proving one of the theorems listed in the book Learning with Kernels:
In particular, it is assumed that $\mu(X) < \infty$, I know that $T_k$ is a compact self-adjoint operator on ...
- 86
0
votes
0
answers
34
views
Prove the uniqueness of this bounded linear operator
I'm reading the paper
Charles R. Baker. Joint measures and cross-covariance operators. Transactions of the American Mathematical Society, 186:273–289, 1973.
I'm stuck in the last part of the proof of ...
- 61
0
votes
0
answers
54
views
Inversion Formula Gaussian Convolution
I am looking at the following 2004 paper by S. Saitoh, called "Approximate real inversion formulas of the Gaussian convolution":
https://www.researchgate.net/publication/...
1
vote
1
answer
31
views
Prove that $f$ is a linear combination of the functions $k_{y_i}(x) = min\{x,y_i\}$.
This is exercise 2.4 of "An Introduction to the Theory of Reproducing Kernel Hilbert Spaces" by Paulsen, and it states:
Let $y_0 = 0 < y_1 < \dots < y_n$ be given and let $f: [0,\...
- 195
1
vote
1
answer
34
views
For given Pontryagin space the norm induced topology is not depending on the fundamental decomposition
For given Hilbert space $H$ consider its corresponding antispace, i.e. the vector space endowed with its negative inner product. $\Pi$ is now called a Pontryagin space if it can be written as a direct ...
