0
$\begingroup$

I am experimenting with support vector machines (SVM) following this book

Without a kernel, it is very easy to "summarize" the SVM optimal solution, as you only need the separator hyperplane w, equal in dimensionality to the number of features considered

when using a kernel on the other hand (e.g. RBF), the real separation is in a higher dimensionality space, so to classify an unseen sample you don't have a simple hyperplane; instead you need to store all the support vectors (part of the training set) and their langrangean multipliers, which is substantially more "complicated".

In the book mentioned above there are some plots that map the separating plane down to the original low dimensionality feature space, like this

enter image description here

I wonder how the author came up with this curve? Is there any procedure for mapping the solution to the 2D space or did he just sample the separating function to draw up the boundary "manually"?

$\endgroup$
1
$\begingroup$

I am the author of the book.

I found the code I used to generate this curve. I think I modified the code from this example

def plot_decision_boundary(axe, classifier):
  h = .02  # step size in the mesh
  x_min, x_max = 0, 20
  y_min, y_max = 0, 20
  xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                       np.arange(y_min, y_max, h))

  Z = classifier.predict(np.c_[xx.ravel(), yy.ravel()])
  Z = Z.reshape(xx.shape)

  cm = plt.cm.prism
  axe.contourf(xx, yy, Z, cmap=cm, alpha=.3)

As you can see, this code use meshgrid which means your second intuition was true. The predict function is used on every data points and then this data is reshaped before being fed to the contourf.

As far as I know, there is no other way to plot the decision boundary when using kernels.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.