As I understood this is a simple full description of SVM algorithm:

There are set of elements (mathematically points). These elements describe as ordered pairs of Cartesian product of two sets X and Y. The approach is to draw a line in the "plane" X x Y such that:

1.1. The points from different classes are on opposite sides of a given line

1.2. The parameters are chosen so direct that maximized minimum distance points to the line

If the line fails to separate original points in original coordinated.

Then let's made:

2.1 Bijective change of coordinates of points using a suitable nonlinear transformation in trying to find the line in the new coordinates

2.2 If we find such tranformation, then when inverse transformation to the original coordinates we can transform out line

2.3 If we still have problem to find such line, we can modify optimization criteria to allow perform some mistakes and weight this mistakes


I have a bunch of question. I'll be glad in answer in any of them:

q1. Why guys from machine learning (some of them) use so complicated terminology like in Russian wiki about SVM?

I think that they are familiar with such concepts as cartesian product of two arbitary sets, and they familiar with 'line' concept? I really don't understand the reason of extra sophistication explanation.

q2. I used term 'line'. I do not understand why they use term 'hyperplane'. This word is 10/4 longer. And such term frightens for me.

q3. Why to guys from machine learning create extra definition of 'kernel trick'. They called step 2.1 with this name. In counting methods 'nonlinear change of coordinates' is called as 'nonlinear change of coordinates'. Why to introduce new term, which beside give wrong hint about convolution kernel?

q4 So I don't see any information in SVM from english wiki about algebra in which it was defined. It is step number "0" in any mathematical sub field to define objects with which you "natively" working.

Why guys from machine learning pass this step?


1 Answer 1


Q2. The term hyperplane is preferred in this sort of classification problem because one frequently wants to work with high dimensional data. When working in a $2$-dimensional space, "dividing the space in two" means finding a line ($1$-dimensional affine subspace) that separates your two groups of points. When working in a $d$ dimensional space, we want to look for a $(d-1)$ dimensional affine subspace, i.e. a plane or hyperplane for $d = 3$ or $d \geq 4$. A hyperplane is the most general way to refer to it in arbitrary $d$.

Q1, Q3. Historically, machine learning has developed from computer science (artificial intelligence, neural networks, etc.) It's only within the last 20 or so years that it has become heavily intertwined with statistics. Hence, a lot of the terminology of ML is different even from other fields of statistics. It's less "introducing unnecessary new terms" and more that terms in ML developed on their own, and by the time the field reached the level of mathematical/statistical sophistication it has today people in the field were uninterested in changing terminology to match that of mathematicians.

Q4. Once again, I think it helps to remember that this is a field that was started by engineers and computer scientists. Historically, they're working with different ideas of mathematical rigor - the first instinct of an engineer or a statistician is not to consider the algebra in which he or she is working.

Generally, I think it would be a mistake to consider ML a mathematical sub-field. There may be mathematical ways of formulating and understanding algorithsm from ML, but the people working on it are statisticians and computer scientists. Thus, many of the assumptions and defaults from mathematics don't carry over.

  • $\begingroup$ q1 -- X x Y -- please take X as R^n and take Y as R^m. Please take X as L2 functions and Y as scalars...if you want....I disagree with you. It is just a line...To be honest I don not here such concept like "high dimensional" in math... It exist concept like finite "dimensional space" and "inifinite dimensional space..." $\endgroup$ Nov 11, 2015 at 15:53
  • $\begingroup$ "It is just a line.." there is no difference, the aim is just not create extra terms. I do not understand why you mentioned "affine space" and more over I don't know fields, except geometry which work with such concept....And it least if you as for X and Y cartesian power of real numbers then for task setup and task solving you need not to use translate at all....I think you would like to say about Linear subspace. $\endgroup$ Nov 11, 2015 at 16:07
  • $\begingroup$ q4 - It's cool but ML use terminology from linear algebra, and probility and finally you need to solve optimization problem via counting methods.The answer is what is x and y elements of which space. Can it be some functions like from L1, L2, Schwartz, Support functions or it should be elements of space with finite dimension? I can imagine line in X x Y space, or "hyperplane", but I'd like to know the generality....Short question -- is space in which the task is solving is linear (under the real number field) $\endgroup$ Nov 11, 2015 at 16:13
  • $\begingroup$ Really all engineers which are involved in Natural Sciences should use math. Just check article of "The Unreasonable Effectiveness of Mat hematics in the Natural Sciences" by EUGENE P. WIGNER, 1959. But in any case you can imagine that this is not math field. But it will be better to be more precisely use derivations. But you should draw boundaries when method is applied. $\endgroup$ Nov 11, 2015 at 16:18
  • $\begingroup$ p.s. In any case big thanks for answers, but I can not accept them, at least my mind can not accept them currently. $\endgroup$ Nov 11, 2015 at 16:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .