1
$\begingroup$

In the image below, the hyperplane is expressed as a dotted line with equation $$\vec{w}.\vec{x} + b = 0.$$

What are the intuitions behind the equation above and finding the hyperplane? And what does the intercept $b$ mean?

To me, it looks like the hyperplane is the middle line with a zero mean, so the intercept term $b$ is zero, but I am not really sure.

enter image description here

$\endgroup$

1 Answer 1

3
$\begingroup$

I always find this easier to think in terms of a vector on the plane. Suppose we have some point on the plane, $x_0$, and we consider some other point on the plane $x$, then this gives a vector $x-x_0$. $w$ is then a vector (a normal vector of the plane) such that, no matter which $x$ we chose, it is perpendicular to $x-x_0$, i.e.,

$$ w \cdot (x-x_0) = 0$$

(e.g., check out the diagram in Hyperplane from vector)

But points in $\mathbb{R}^n$ are really just vectors from $0$ to a point, so we may rewrite this as

$$ w\cdot (x - x_0) = w \cdot x - w\cdot x_0 = w\cdot x + b = 0$$ where we define $b= - w\cdot x_0$. So $b$ is essentially playing a higher dimensional role as the $x$-intercept for a line: it allows the plane to move away from the origin.

Now $w$ is unique up to constant multiple (in other words we may stretch this vector above or below the plane), so we scale it so that the hyperplane is centered between points that it is separating. We scale so that $w\cdot x + b = 1$ and $w\cdot x + b = -1$ are the margin planes. Why $\pm 1$? Why not? We could just as well use $w\cdot x + b = 10$ and $w\cdot x + b = -10$, this will just change $b$ and the ultimate resulting $w$.

Finally, we may calculate the distance between these two margin planes, which is given by $\frac{2}{\|w\|}$. Hence, if we intend to maximize the separation of the margin planes, we do so by minimizing $\|w\|$. That is SVM construction.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .