2
$\begingroup$

I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm says the following:

We are now in the position to describe a pattern recognition learning algorithm that is arguably one of the simplest possible. We make use of the structure introduced in the previous section; that is, we assume that our data are embedded into a dot product space $\mathcal{H}$. Using the dot product, we can measure distances in this space. The basic idea of the algorithm is to assign a previously unseen pattern to the class with closer mean.

We thus begin by computing the means of the two classes in feature space; $$\mathbf{c}_+ = \dfrac{1}{m_+} \sum_{\{ i \vert y_i = +1 \}} \mathbf{x}_i, \tag{1.7}$$ $$\mathbf{c}_- = \dfrac{1}{m_-} \sum_{\{ i \vert y_i = -1 \}} \mathbf{x}_i, \tag{1.8}$$ where $m_+$ and $m_-$ are the number of examples with positive and negative labels, respectively. We assume that both classes are non-empty, thus $m_+, m_- > 0$. We assign a new point $\mathbf{x}$ to the class whose mean is closest (Figure 1.1). This geometric construction can be formulated in terms of the dot product $\langle \cdot, \cdot \rangle$. Half way between $\mathbf{c}_+$ and $\mathbf{c}_-$ lies the point $\mathbf{c} := (\mathbf{c}_+ + \mathbf{c}_-)/2$. We compute the class of $\mathbf{x}$ by checking whether the vector $\mathbf{x} - \mathbf{c}$ connecting $\mathbf{c}$ to $\mathbf{x}$ encloses an angle smaller than $\pi / 2$ with the vector $\mathbf{w} := \mathbf{c}_+ - \mathbf{c}_-$ connecting the class means. enter image description here

It is this part that I'm confused by:

We compute the class of $\mathbf{x}$ by checking whether the vector $\mathbf{x} - \mathbf{c}$ connecting $\mathbf{c}$ to $\mathbf{x}$ encloses an angle smaller than $\pi / 2$ with the vector $\mathbf{w} := \mathbf{c}_+ - \mathbf{c}_-$ connecting the class means.

The angle denoted in Figure 1.1, as far as I can tell, is not the angle "enclosed" by $\mathbf{x} - \mathbf{c}$ and $\mathbf{w}$, and is actually the opposite angle. Am I misunderstanding this, or does this seem to be an error in the image?

The figure is also available in this research paper (Figure 1) by the same authors.

$\endgroup$
4
  • $\begingroup$ I may be misunderstanding what you're saying you're misunderstanding, but ... If you are saying that the angle marked with the dot in the figure isn't the one described, then you're correct. The dot-mark, an alternative to the "square mark", indicates that the dashed line/hyperplane is perpendicular to vector $w$. This provides a visual reference for comparing the described angle (determined by $x$ and $w$) to $\pi/2$. (As shown, that angle is larger.) $\endgroup$
    – Blue
    Commented Dec 31, 2020 at 1:11
  • $\begingroup$ @Blue Yes, that is indeed what I was alluding to. So the referenced "enclosed" angle is the angle where the vector arrow for $\mathbf{c}$ is pointing to? Or do I have this wrong? $\endgroup$ Commented Dec 31, 2020 at 12:14
  • 1
    $\begingroup$ The "enclosed" angle is bounded by the (thick) arrowheads. If $X$, $C$, $C_+$ are the points at the ends of vectors $x$, $c$, $c_+$, then that angle is $\angle XCC_+$. $\endgroup$
    – Blue
    Commented Dec 31, 2020 at 12:24
  • 1
    $\begingroup$ @Blue Great, thanks for taking the time to clarify this! (Feel free to post that as an answer if you want.) $\endgroup$ Commented Dec 31, 2020 at 12:25

1 Answer 1

0
$\begingroup$

Converting comments to an answer ...

OP has confirmed misinterpreting the angle marked with the dot as the angle described as being "enclosed" by the vectors. However, that dot-mark, an alternative to the "square mark", is meant to indicate that the dashed line/hyperplane is perpendicular to vector $w$. This provides a visual reference for comparing the described angle (determined by $x$ and $w)$ to $\pi/2$.

The "enclosed" angle is in fact the one bounded by the thick arrowheads. Writing $X$, $C$, $C_+$ for the ends of vectors $x$, $c$, $c_+$, this is the angle $\angle XCC_+$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .