3
$\begingroup$

When reading about the optimization problem for Support Vector Machine in Bishop's book (Pattern Recognition and Machine Learning) he wrote that:

The optimization problem then simply requires that we maximize $\| \mathbf w\|^{-1}$ which is equivalent to minimizing $\| \mathbf w\|^{2}$

Here $\| \mathbf w\|$ is the euclidian norm of the vector $\mathbf w$ so it is always a positive number.

I am wondering why do we not minimize $\| \mathbf w\|$ instead of $\| \mathbf w\|^2$ ?

Does using $\| \mathbf w\|^{2}$ simplifies the optimization problem?

At first I thought that taking the square made the problem convex, but the function $$f(x,y)=\sqrt{x^2+y^2}$$ looks convex to me too.

$\endgroup$
1
  • 3
    $\begingroup$ Minimizing $\lVert\cdot\rVert$ and minimizing $\lVert\cdot\rVert^2$ are the same thing, aren't they? $\endgroup$
    – Clement C.
    Jun 6, 2016 at 13:04

1 Answer 1

5
$\begingroup$

Minimizing $|w|$ is the same as minimizing $|w|^2$. As I understand it, the reason we choose the latter is purely for convenience. Would you rather deal with partials of $\sqrt{x^2+y^2}$ or $x^2+y^2$? The latter, obviously, because it's easier.

$\endgroup$
1
  • 1
    $\begingroup$ And its derivative is well defined. $\endgroup$
    – Royi
    Jun 9, 2016 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.