I have recently started learning about machine learning and have come across kernels and null spaces. I understand that null space is the set of all vectors that satisfy the equation A.v = 0 (Where A is a matrix). I have been taught that null space is a set of vectors that are squished to 0 when transformation matrix A is applied. Then I came across SVM where kernel functions are used. I read that null spaces are also called kernels of a matrix. My questions are as follows.

  1. Are both kernel functions and null spaces same? If they are related, how are they related?
  2. What is the relation between kernel functions used in SVM and null space of matrix? If yes, how? Is this derived from the null space of transformations used in SVM?
  3. What is the reasoning behind kernels used in kernel convolution? For example: Is the Gaussian kernel used a representation of the transformation? How are they related to null spaces? If they are related, how are they related?

I have already asked this in signal processing site and CV site. But, I haven't received any answer yet. Could you please answer these questions? I am very confused. Thank you in advance.

  • 1
    $\begingroup$ The two uses of the word "kernel", I believe, are generically not related to one another. The word "kernel", as a real old English word, is something like: the core of something (like the edible nut inside of the hard shell). One can understand "kernel" as a thing/concept around which everything is built (this interpretation is the motivation for the linear algebra definition). One can also understand "kernel" as the essential thing connecting everything together (this is the motivation behind the SVM definition). $\endgroup$ – Hamed Jul 12 '18 at 15:17
  • 1
    $\begingroup$ Another motivation, for usage of the same word "kernel" in two contexts, might be from one of the first times a kernel function appeared in mathematics. An example, would be: Consider the inhomogeneous differential equation $y''+by'+cy=g(x)$ and let $K(x)$ be the solution to homogeneous version $y''+by'+cy=0$ (which in a sense is kernel, in the linear algebra sense). Then $\int K(t-s)g(s)ds$ is the solution to the inhomogeneous DE (in this integral $K$ is the kernel, as the weighting function, similar to SVM). $\endgroup$ – Hamed Jul 12 '18 at 15:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.