support vector machine - vector algebra

Question

I am trying to understand the basics of SVM algebra, but fail to understand per below:

Let us formalize an SVM with algebra. A decision hyperplane can be defined by an intercept term $b$ and a decision hyperplane normal vector $\vec{w}$ which is perpendicular to the hyperplane. This vector is commonly referred to in the machine learning literature as the weight vector . To choose among all the hyperplanes that are perpendicular to the normal vector, we specify the intercept term $b$. Because the hyperplane is perpendicular to the normal vector, all points $\vec{x}$ on the hyperplane satisfy $\vec{w}^{T}\vec{x} = -b$.

can someone give me a pointer / hint to understand the last equation? is it linked to the fact that the cross-product of two perpendicular vectors should be zero? much appreciated

Answer 1

A line is a hyperplane in the 2-dimensional space. Think of your equation as an equation defining a line in your space (simple as that: $a\bullet x+b=0$). Now, you can see that your $w$-vector is the slope of your line/decision boundary and your equation is a way to create a good decision boundary between two classes in the 2-dimensional space. Think of $w$ as the force with which we can push the decision boundary to different directions.

It is true, as seem to already know, that $w\bullet x$ is the inner or scalar product defined as $w\bullet x = \sum_{i}{w_ix_i}$.

In the SVMs, we can say that the decision boundary is used by saying that any $x$ value that gives a positive result for $wx+b$, is above the line, and so it belongs to the '+' class, while any $x$ that gives a negative result is in the '-' class.