In the book elements of statistical learning (e2 pg 130) it gives the definition of a hyperplane or affine set $L$ by the equation:

$$ f(x)=\beta_0+\beta^{\text{T}}x=0 $$

and lists some properties about hyperplanes / affine sets:

  1. For any two points $x_1$ and $x_2$ lying in $L$, $\beta^\text{T}(x_1-x_2)=0$ and hence $\beta^*=\beta/\lvert\lvert\beta\rvert\rvert$ is the vector normal to the surface of $L$

  2. For any point $x_0$ in $L$, $\beta^\text{T}x_0=-\beta_0$

  3. The signed distance of any point $x$ in $L$ is given by

$$ \begin{align} \beta^{*\text{T}}(x-x_0) &= \frac{1}{\lvert\lvert\beta\rvert\rvert}(\beta^\text{T}x+\beta_0)\\ &=\frac{1}{\lvert\lvert f'(x)\rvert\rvert}f(x) \end{align} $$

To my understanding the difference between an affine set and a hyperplane is that the latter passes through / contains the origin and the former is a translation from the origin.

Given the equation defining an affine set, it is clear to me that point 2 must hold. However, I do not see why one can derive that $\beta^*=\beta/\lvert\lvert\beta\rvert\rvert$ is the vector normal given $\beta^\text{T}(x_1-x_2)=0$. Nor do I understand how the norm of $\beta$ is the first derivative of $f$.

Can someone please elaborate?


Let us refer to the figure below, from the book you mentioned. Let us define the hyperplane $L$ using: $$f(x)=\beta_0+\beta^Tx=0$$

If two points $x_1$ and $x_2$ lie in $L$, then they both satisfy the equation above, i.e.: $$\beta_0+\beta^Tx_1=0\\ \beta_0+\beta^Tx_2=0,$$ which gives you, if you subtract these two: $$\beta^T(x_1-x_2)=0.$$ You can interpret this as a dot product equal to zero indicating that $\beta$ is perpendicular to $(x_1-x_2)$ and since this latter is a vector that lies in $L$ this means that $\beta^*=\displaystyle\frac{\beta}{||\beta||}$ is normal to $L$.

Now, if $f(x)=\beta_0+\beta^Tx=0$, then the derivative $f'(x)=\beta$, and $||\beta|| = ||f'(x)||$.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.