6
$\begingroup$

I see the terms kernel and distribution used - what I presume to be - interchangeably all the time and hence my understanding is that they are the same e.g. within a publication the phrase "a gaussian kernel" and "a gaussian distribution" appears - to me - synonymous.

However, it is possible, as can be the case, that some nuance is missing from this comparison.

In both Meaning of "kernel" and What does kernel mean no definite answer is given. What are the most overloaded words in mathematics highlights that often terms in mathematics may have non-unique meanings - especially across fields.



So is there a formal or otherwise distinction between a kernel and a distribution?

Or is a kernel just any symmetric function that integrates to 1? i.e.

$$K(-u) = K(u)$$ and $$\int\limits_{-\infty}^\infty K(u)\mathbb{d}u=1$$

Notably if $K(-u) = K(u)$ then any tailed distribution (e.g. Weibull, Gamma, etc) is not a kernel?



Further befuddlement stems from articles like this, stating that:

kernel density estimation (KDE) is a non-parametric way to estimate the probability density function of a random variable.

Such phrasing is, again, symmetric and - to me - implies that if a kernel estimation estimates a probability function, then a tried-and-true kernel is a probability function.



If kernels are not , what is a good / accessible resource to clarify my confusion?

$\endgroup$
5
  • $\begingroup$ Different things in mathematics have been called kernels with different intentions. Can you clarify the context in which you came across kernels? $\endgroup$ Mar 1, 2017 at 14:35
  • $\begingroup$ @TenaliRaman I have come across the term kernel in a lot of different environments, which is perhaps part of my confusion. Most commonly I see the term kernel used in regards to probability, statistics, algorithmics, and machine learning. It would be cool to get a clear distinction between these fields. $\endgroup$
    – SumNeuron
    Mar 1, 2017 at 14:41
  • $\begingroup$ Maybe this wikipedia link might help, if you haven't already seen it. $\endgroup$ Mar 1, 2017 at 14:43
  • $\begingroup$ @TenaliRaman unfortunately that isn't particularly useful, as - I have already see it - and reading the various articles have convolved their meanings in my head even further e.g. kernel (statistics) vs stochastic kernel, etc $\endgroup$
    – SumNeuron
    Mar 1, 2017 at 14:46
  • $\begingroup$ @TenaliRaman for now, lets go with kernel (statistics). Tangentially - kernels used in K.D.E. have a bandwidth parameter, is there something analogous for distributions? $\endgroup$
    – SumNeuron
    Mar 1, 2017 at 14:48

1 Answer 1

4
$\begingroup$

Probably it is too late to answer that but as I saw this unanswered and I think I can help.

The term distribution refers to the theoretical and unknown function that explains the behavior of a random variable. Normal, Gamma, Weibull are all well known distributions. In statistics and probability the kernels are ways to estimate a distribution.

A gaussian kernel and a gaussian distribution are two different things. The gaussian distribution is defined as $$f(x)=\frac{1}{\sigma\sqrt{2\pi}}exp\left(-\frac{(x-\mu)^{2}}{2\sigma^{2}}\right)$$.

The kernel density estimator is defined as

\begin{eqnarray} \hat{f}(x)=\frac{1}{nh}\sum_{i=1}^{n}K\left(\frac{x-X_{i}}{h}\right), \end{eqnarray}

where $X_{i}$ $i=1,2,\ldots, n$ is your sample. Note the hat above $f$. This is simply a way of estimating the true and generally unknown $f$ that could be a gaussian density , a weibull density or whatever. Now regarding $K(\cdot).$ This is the so called kernel function and this function is taken to be a density itself. For example you can define $K(\cdot)$ to be the standard normal density, i.e. this is called the gaussian kernel. But you can also use other choices such as the triangular kernel, the Epanechnikov etc.Using the kernel density estimator shown above , even if you define $K(\cdot)$ to be a gaussian kernel, you can estimate very efficiently bimodal or even multimodal distributions.

$\endgroup$
3
  • 2
    $\begingroup$ That is not the only usage of "kernel" even just in probability. $\endgroup$
    – Ian
    Sep 19, 2017 at 2:49
  • 1
    $\begingroup$ Is there another usage of the terminology "gaussian kernel" in statistics (that is used as density estimator, integrates to one, and is symmetric as the original post illustrates) that I might be missing? $\endgroup$
    – leo
    Sep 19, 2017 at 2:53
  • 1
    $\begingroup$ One example is in stochastic processes, where you have a transition kernel, which for a Gaussian process is referred to as a Gaussian kernel. The usage of the term "convolution kernel" from analysis probably bleeds into probability language sometimes as well. You probably have the usage the OP meant, however. $\endgroup$
    – Ian
    Sep 19, 2017 at 13:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .