# SVM : Why can we set 1 in the hyperplane equation?

I am reading the Wikipedia article about SVM and there is something I don't understand. When they say:

These hyperplanes can be described by the equations

$$wx - b=1$$ and

$$wx - b=-1$$

I was wondering where does the +1 and -1 come from?

I found two papers which explain that this is an arbitrary choice:

We can write the following equations for the support hyperplanes:

$$w^T x = b + \delta$$ $$w^T x = b − \delta$$ We now note that we have over-parameterized the problem: if we scale w, b and $\delta$ by a constant factor $\alpha$, the equations for x are still satisfied.

To remove this ambiguity we will require that $\delta$ = 1, this sets the scale of the problem, i.e. if we measure distance in millimeters or meters

Source

but I don't understand what he means when he says "this sets the scale of the problem"

and

Note that if the equation $f(x) = wx + b$ defines a discriminant function

(so that the output is > $sgn(f(x))$),

then the hyperplane $cwx + cb$ defines the same discriminant function for any $c > 0$.

Thus we have the freedom to choose the scaling of $w$ so that $min_{x_i} |wx_i + b| = 1$.

Source

but I don't understand why he introduces $min_{x_i} |wx_i + b| = 1$.

My understanding is that we can do it because variables $w$, $b$ and $\delta$ are kind of linked together.

Can we change the Wikipedia definition and say

These hyperplanes can be described by the equations $$wx - b= 2$$ and $$wx - b=- 2$$

or is this incorrect and so we must say that :

These hyperplanes can be described by the equations $$2wx - 2b= 2$$ and $$2wx - 2b=- 2$$

Could you clarify this for me?

## 1 Answer

Very simply, if you choose any number other than 1, you can simply scale it away again. Consider $$\mathbf{w}\cdot\mathbf{x}-b=\pm\delta$$ Now, divide both sides of the equation by $\delta$, and we get $$\left(\frac1\delta\mathbf{w}\right)\cdot\mathbf{x}-\frac{b}{\delta}=\pm1$$ Which means that we can define $\mathbf{\hat w}=\frac1\delta\mathbf{w}$ and $\hat b=\frac{b}{\delta}$, and we have $$\mathbf{\hat w}\cdot\mathbf{x}-\hat b=\pm1$$ And because the goal is to minimise $\|\mathbf{w}\|$ (in order to maximise the size of the margin, $\frac2{\|\mathbf{w}\|}$), it doesn't matter if we scale $\mathbf{w}$ by some constant such as $\delta$ first - and so, we can use $\mathbf{\hat w}$ in place of $\mathbf{w}$, and the choice of $\delta$ is irrelevant (aside from needing to be a positive real number).

Since it's irrelevant, might as well make it the simplest possible positive real number, 1.

As for why it "sets the scale" of the problem, think of it this way: changing $\delta$ would change the scaling of $\mathbf{w}$ (that is, choosing $\delta=2$ would make $\mathbf{w}$ twice as big, for example). And so, just as changing $\delta$ changes the scale, so too does setting $\delta$ set the scale - it keeps it fixed, rather than having it vary from instance to instance.

• Thanks Glen....! – Pugl Mar 29 '18 at 13:19
• Perfect! You cleared my confusions. +1 – Kaushal28 Apr 1 '19 at 8:28