A lot of material on the web regarding Loss functions talk about "minimizing the Hinge Loss".
However, nobody actually explains it, or at least gives some example. The best material I found is here from Columbia, and I include some snippets from it below.
I understand the hinge loss to be an extension of the 0-1 loss. The 0-1 Loss Function gives us a value of 0 or 1 depending on if the current hypothesis being tested gave us the correct answer for a particular item in the training set. The hinge loss does the same but instead of giving us 0 or 1, it gives us a value that increases the further off the point is.
This formula goes over all the points in our training set, and calculates the Hinge Loss $w$ and $b$ causes. It sums up all the losses and divides it by the number of points we fed it.
This much makes sense to me.
What's confusing me is as follows:
- How do you plot a hinge loss function?
- How do you minimize it? Isn't the minimal always zero?
- How should I understand the typical hinge loss graph? Are they just gross oversimplifications? For example, the green line represents the hinge loss function you see in every image in a Google search for "hinge loss".
**Please, **Can someone provide (for the world) a simple example of hinge loss minimization? Let's say I have four negative points (blue circles) and four positive points (red squares). What would the loss function look like? How do I minimize (mathematically, and with intuition).
Knowing this would be a huuuge help for me, and probably for many others, as the resources on this popular topic are scarce. Thanks!