Questions tagged [reproducing-kernel-hilbert-spaces]
The reproducing-kernel-hilbert-spaces tag has no usage guidance.
1
vote
0answers
12 views
Re-expressing cost function with a kernel
I need to re-express a cost function I will eventually optimize in terms of the provided exponential kernel.
The cost function $q(u)$, where $u \in [0, U]$, belongs to the reproducing kernel hilbert ...
0
votes
0answers
17 views
Gradient of square norm in RKHS
Consider a RKHS $\mathcal H$, with continuous reproducing kernel $K$. I am confused regarding the gradient of $\tfrac 12 \Vert \cdot \Vert_{\mathcal{H}} ^ 2$.
On the one hand, I'd expect it to be ...
0
votes
0answers
11 views
Find a bound of the biggest radius of hypersphere in convex hull centered on centroid
Setting:
We have a probability distribution on a space $\mathcal{X} \subset \mathbb{R}^d$, called $\rho(x)$, and we are given a sample of iid points $S^n = \{x_i\}_{i=1}^n$ from $\rho$.
Let $K: \...
1
vote
1answer
35 views
Kernels and finite maps
Let $\bf{x}, \bf{z} \in \mathbb{R}^n$. If the Gaussian kernel is defined by:
$$K(\bf{z}, \bf{x}) = \exp\left( - \frac{\|\bf{z} - \bf{x}\|_2^2}{\sigma^2}\right) $$
I'd like to know if there is a ...
0
votes
0answers
8 views
RKHS for matrix valued input.
The question is related to the link : https://en.wikipedia.org/wiki/Reproducing_kernel_Hilbert_space
As I understand the Mercer theorem we can get RKHS. This lead to the kernel trick as mentioned in ...
0
votes
0answers
6 views
SVM: Are kernel functions closed under positive real powers?
If k is a valid kernel (meeting Mercer's conditions) is $k^{\alpha}$ for $\alpha \in \mathbb{R}^{+}$ a valid kernel in general?
Any references showing a proof or otherwise would be much appreciated?
0
votes
0answers
29 views
Kernel function with a feature space equipped with an inner product that is not the dot product
Premise:
A function $K: \mathbb R^d \times \mathbb R^d \to \mathbb R$ is called
a kernel function on $\mathbb{R}^d$ if there exists a Hilbert space
$\mathcal{H}$ and a map $\phi: \mathbb R^d \...
0
votes
0answers
6 views
what does it mean? "the function class becomes an approximation to the Reproducing Kernel Hilbert Space corresponding to the Gaussian kernel.
It seems like embedding. But I am not sure what approximation to RKHS means. Or how to prove something is approximation to RKHS?
0
votes
1answer
18 views
Reducing Kernel Hilbert Space: Reproducing property
If the inner product between tho functions is the integral over R of f(x)g(x)dx and it's equal to f(x), how whould g(x) be in order to satisfy this equality?
0
votes
0answers
18 views
Reproducing Kernel Hilbert Spaces
I don't undestand why:
The set of continuos functions from a metric space X→R, C(X), forms a vector space over R using the usual definitions of addition and scalar multiplication.
0
votes
0answers
12 views
How to decide the weight of the locally logistic regression?
I have a problem of how to decide the weight of the logistic regression. My model is as below.
For the general logistic regression, we have the likelihood of the model:
\begin{align}
L(\theta) &=...
0
votes
1answer
24 views
Simple (?) Quesiton on Inner Product in Reproducing Kernel Hilbert Space
I'm following the gentle introduction to Reproducing Kernel Hilbert Spaces From Zero to Reproducing Kernel Hilbert Spaces in Twelve Pages or Less by Hal Daum\'e III. I believe the author fully ...
0
votes
1answer
62 views
How to prove that the Gram matrix of the Gaussian kernel has full rank?
I'm trying to prove that, given mutually different points $x_1,\dots,x_m$, the Gram matrix $G$ for the Gaussian kernel has $rank(G)=m$. If I can prove that the Gaussian kernel is strictly positive ...
0
votes
0answers
16 views
What is the meaning of stochastic sampling?
I came across this term in the context of Kernel Methods for Supervised Learning.
Subsampling is the selection of a subset from the training set. But what is stochastic subsampling, I understand that ...
0
votes
0answers
29 views
RKHS of a Polynomial Kernel with negative roots
Wikipedia, and actually all books and ressources I could find, define a polynomlial kerenel as:
$$
K: x,y \mapsto (x^Ty + c)^2,
$$
where $c\ge 0$. What happens if $c < 0$? Take the following ...
0
votes
1answer
25 views
Homogeneous Fredholm Integral Equation of Second Kind with Boundary Conditions
I have come across a claim that I am quite certain is true, but would be benefited from seeing a proof sketch.
Suppose we know the following equation:
$$
f(z)=\int_0^T K(z,x)f(x)\,\mathrm{d}x,
$$
...
0
votes
1answer
69 views
Each Hilbert space of holomorphic functions on $\mathbb C$ is a reproducing kernel Hilbert space?
I would like to know is the following result true ?
"Each Hilbert space of holomorphic functions on $\mathbb C$ is a reproducing kernel Hilbert space".
Where a Hilbert space of holomorphic ...
0
votes
0answers
61 views
Uniqueness of Reproducing Kernel Hilbert Space
Digging in the definition of Reproducing Kernel Hilbert Spaces (RKHS) I came across the following example taken from pages 49-51 of [1]:
Given the kernel $k(x,y) = \langle x,y\rangle^2$, with $x,y\in ...
1
vote
1answer
26 views
A reproducing kernel Hilbert Space. Barry Simon problem 4.
A reproducing kernel Hilbert Space is a Hilbert space , $H$, of functions , $f$ on a set $E$, so that
i) For any $f$, there is $x\in E$ with $f(x)\not=0$
ii) For any $x\in E$, there is $f\in H$ so ...
0
votes
0answers
54 views
Does the Gaussian RKHS contain a nonzero constant function?
I am working with the reproducing kernel Hilbert space $\mathcal{H}$ of functions generated by the Gaussian kernel $k(x,y)=\exp{\left(-\frac{1}{2\sigma^2}|x-y|^2\right)}$ over an interval $[-1,1]$. Is ...
0
votes
0answers
12 views
Correspondence between RKHS and Gaussian process realzation
Consider
* the space of realizations of Gaussian process (GP) with zero mean and kernel (covariance function) $k(x, x')$
* the space of functions from RKHS generated by the same kernel $k(x, x')$.
...
0
votes
0answers
22 views
Why $\nu>-1$ in weighted Bergman spaces?
Let $\Omega$ be a non-empty connected open subset of $\mathbb{C}$, let $\mu$ denote the Lebesgue measure on $\Omega$ and let $H(\Omega)$ denote the set of holomorphic functions defined on $\Omega$.
...
0
votes
0answers
10 views
How to prove that the inner space has this property?
Let $A=[a_{i,j}]_{n\times n}$ be a real positive semi-definite type $n\times n$ matrix whose columns are $\mathbf{a}_1,\mathbf{a}_2,\ldots,\mathbf{a}_n$. Let those columns span $V \subset R^n$. How ...
0
votes
0answers
12 views
How to prove this equality relating to Fourier-transform and Reproducing-Kernel-Hlibert-Space?
How can we prove following equality? (or, is this equality valid?)
Let $x$ and $\zeta$ be members of $\mathbb{R}^d$.
And let $K$ be kernel function, that is, $K$ is a symmetric positive-definite ...
0
votes
0answers
15 views
Kernel function from Hausdorff distance
Is there a way to build a positive definite kernel on the space of arbitrary point-sets in $\mathbb{R}^d$ using the Hausdorff distance?
I tried the obvious option of setting $K(X, Y) = \exp\{-d_H(X, Y)...
0
votes
0answers
24 views
Derivation of Kernel Function
I'm strugging to find information on how to derive a kernel function or the “feature space”. Let’s assume we have a matrix $x$ of dimension $(n,p)$. We have $x_i^j$, the $i$-th observation of the $j$-...
1
vote
0answers
47 views
eigenvalues and their analogy with frequency
I was reading this document. At page 25, paragraph 2, there was mention of eigenvalues (and frequency; degree of freedom) of a kernel matrix. I am attaching the corresponding image for reader's ...
1
vote
1answer
125 views
Is there a positive-semidefinite convolution kernel, that is continuous at $0$ but discontinuous elsewhere?
A positive-semidefinite, symmetric convolution kernel on the circle $\mathbb{T}^1$ is a function $k:\mathbb{T}^1\to\mathbb{R}$ such that $k(x)=k(-x)$, and $\sum_{i=1}^n\sum_{j=1}^n k(x_i-x_j)c_i c_j\...
2
votes
0answers
155 views
Linear regression with feature representation confusion - is design matrix column space the feature space?
I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
1
vote
0answers
24 views
Invertibility of Grammian in Reproducing Kernel Hilbert Space
Let $\mathcal{H}$ be a Hilbert space with a reproducing kernel $K:X \times X \rightarrow \mathbb{R}$. For any finite sequence $x_1,...,x_n$ of distinct points in $X$ we define Gramm matrix as
$$
M(x_1,...
0
votes
0answers
39 views
Orthonormal system $\{ e_n(t)\}_{n=1}^{\infty}$ is complete $\Leftrightarrow$ $k(t,t) = \sum_{n=0}^{\infty}{|e_n(t)|^2}, \forall t \in \Omega$
I want to prove that, in a RKHS (Reproducing Kernel Hilbert Space), being k(t,s) its reproducing kernel:
Orthonormal system $\{ e_n(t)\}_{n=1}^{\infty}$ is complete $\Leftrightarrow$ $k(t,t) = \sum_{...
0
votes
0answers
37 views
Are Matérn class kernels universal kernels or not?
This is a question that I can't find the solution. I don't know it is a open question or it is a well-known result that can be attained from several lemmas.
Here are the definition of Matérn class ...
1
vote
0answers
40 views
A biliographic inquiry into Fredholm's kernel
I would like to get the exact bibliographic references from the paper(s) by Erik Ivar Fredholm in which the definition(s) of kernel and trace first appear.
If someone could even copy the extract from ...
1
vote
0answers
33 views
Does the optimal function in kernel method have a sparse representation?
The kernel least square method aims to solve the following problem,
$$f^*=\arg \min_{f\in \mathcal{H}} E_{x,y}[(f(x)-y)^2]+\frac{\lambda}{2}\Vert f\Vert_{\mathcal{H}}$$
, where $\mathcal{H}$ is some ...
0
votes
0answers
20 views
Representer theorem and support vector machines
The objective function of support vector machines can be written as
$\sum_{i=1}^n (1-Y_i(K_i^T\alpha+\mu))_+ +\lambda \alpha ^T K \alpha, $ where K is a kernel matrix, $K_i$ is its ith column. How to ...
1
vote
2answers
48 views
RKHS rough definition
Wikipedia provides an intuitive explanation of RKHS as a Hilbert space of functions where point evaluation is a continuous linear functional:
Roughly speaking...if two functions $f$ and $g$ in the ...
0
votes
2answers
113 views
Is a Reproducing Kernel Hilbert Space just a Hilbert space equipped with an “indexed basis”?
I haven't studied any functional analysis yet. My linear algebra is pretty good, I think.
Consider the tuple $(H, I, \phi)$ where $H$ is a Hilbert space, $I$ is an abstract set, and $\phi:I \to H$ is ...
3
votes
1answer
169 views
Example of an infinite dimensional Hilbert space that is not an RKHS
I have been studying Reproducing Kernel Hilbert Spaces (RKHS). The definition I am using is as follows: An RKHS is a Hilbert space $\mathcal{H}$ of real-valued functions on a set $X$ such that for all ...
0
votes
0answers
79 views
Prove strictly positive-definite kernel
It can be shown that the Hamming distance defines a positive semi-definite kernel (see e.g., here). However, is the kernel
$$ K(\mathbf{x}, \mathbf{y}) = 1 - \frac{1}{n} \sum_{i=1}^n \mathbb{I}\{x_i = ...
3
votes
0answers
150 views
Which theorem tells us that RKHS is dense in L2?
I have heard this phrase quite a lot now, that RKHS is dense in the space of bounded continuous functions ($\mathcal L_2$).
For example, this would be true with
$$f(x) = \sum_{i=1}^N \alpha_i K(x_i,...
1
vote
0answers
54 views
Minimizing Hilbert-Norm through Gradient Descent in Parsimonious Online Learning with Kernels
I am conducting research on an algorithm called Parsimonious Online Learning with Kernels (POLK), which uses reproducing kernel hilbert spaces as a means for approximating a function.
Here is a link ...
2
votes
0answers
66 views
Statistical learning and differentiation in a Hilbert space
Let $\mathcal{H}$ be a Hilbert space and $\Phi\colon\mathbb{R}^d\to\mathcal{H}$ some function with $K$ being the RKHS kernel. Let $B_i>0$ for all $i=1,2,\ldots ,n$ and we are interested in solving ...
3
votes
0answers
71 views
Characterisation of Banach space $B$, given the RKHS of a Gaussian random variable in $B$.
The setting:
Given a Banach space valued Gaussian random variable $X$ (throughout I will assume everything to be centered and separable) we can define its reproducing kernel Hilbert space $$H:=\...
1
vote
1answer
224 views
Completion of span in $C(X)$ (RKHS)
In the construction of Reproducing Kernel Hilbert spaces via the Moore–Aronszajn theorem one uses the completion of the linear span of $\{K_x |\ x\in X\}$, where $K_x(y)=K(x,y)$ and $K$ is some ...
1
vote
1answer
109 views
Why is $\mathbb{R}^3$ not the Reproducing Kernel Hilbert Space defined by kernel $k(x,y)=(x_1y_1+x_2y_2)^2$
Let the input space be $\mathcal{X}=\mathbb{R}^2$, and kernel function $k(x,y)=\langle x,y \rangle^2 = (x_1y_1+x_2y_2)^2$.
I can write
\begin{align}
k(x,y) &= x_1^2 y_1^2 + x_2^2 y_2^2 + 2 x_1 ...
0
votes
1answer
125 views
Linear kernel is strictly positive definite, but corresponding RKHS in not dense in C(R)?
There must be something very basic that I am missing.
If I understood correctly from wikipedia, positive-definiteness of kernel $K(\cdot,\cdot)$ is sufficient for universality of this kernel. Then, ...
2
votes
1answer
297 views
Reproducing kernel vs. Riesz kernel.
I'm attending an Analysis course and we are studying the Hardy space $H^2$ in the unit disk, from where the concept of RKHS (Reproducing Kernel Hilbert Space) came out. Acording to my notes:
By the ...
2
votes
0answers
424 views
How to describe the Reproducing Kernel Hilbert Space (RKHS), given a kernel?
So, I have the following exercise I need to solve:
"For $x, y \in \mathbb{R}, \: K_1(x,y) = (xy + 1)^2 \:$ and $\: K_2(x,y) = (xy - 1)^2$. Describe the RKHS of $K_1$, $K_2$ and $K_1 + K_2$."
By "...
