Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a question pertaining to linear separability with hyperplanes in a support vector machine.

According to Wikipedia:

...formally, a support vector machine constructs a hyperplane or set of hyperplanes in a high or infinite dimensional space, which can be used for classification, regression or other tasks. Intuitively, a good separation is achieved by the hyperplane that has the largest distance to the nearest training data points of any class (so-called functional margin), since in general the larger the margin the lower the generalization error of the classifier.classifier.

The linear separation of classes by hyperplanes intuitively makes sense to me. And I think I understand linear separability for two-dimensional geometry. However, I'm implementing an SVM using a popular SVM library (libSVM) and when messing around with the numbers, I fail to understand how an SVM can create a curve between classes, or enclose central points in category 1 within a circular curve when surrounded by points in category 2 if a hyperplane in an n-dimensional space V is a "flat" subset of dimension n − 1, or for two-dimensional space - a 1D line.

Here is what I mean:

circularly enclosed class separation for a 2D binary SVM

That's not a hyperplane. That's circular. How does this work? Or are there more dimensions inside the SVM than the two-dimensional 2D input features?

share|improve this question
1  
Yes, I believe this is something called the kernel trick. Google will find you some nice explanations. –  Rahul Oct 22 '10 at 15:57
add comment

3 Answers

up vote 2 down vote accepted

As mentioned, the kernel tricks embeds your original points to a higher dimensional space (in fact, in some cases infinite dimensional - but of course the linear subspace generated by your actual points is finite dimensional).

As an example, using the embedding $(x,y) \mapsto (x,y,x^2,y^2)$ (that actually corresponds to a quadratic kernel, I think) then the equation of an arbitrary ellipse becomes linear.

share|improve this answer
add comment

Cross-posted my question at StackOverflow. More responses there.

share|improve this answer
add comment

Check out Andrew Ng's website for a tutorial on SVMs. www.stanford.edu/class/cs229/notes/cs229-notes3.pdf

Yeah, in this case I believe that the feature space and what you are representing are two different dimensions.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.