6
$\begingroup$

Finding the vector perpendicular to the plane

Why is weight vector orthogonal to decision plane in neural networks

I am thinking now about hyperplanes and orthogonal vectors. My problem is following one:

Definition of hyperplane: $$H=\left\{ x\ \epsilon\ R^2 : u^Tx=v \right\}$$

Consider for instance simple case of hyperplane in $R^2$ (straight line):

$$H=\left\{ x\ \epsilon\ R^2 : u^Tx=3 \right\}$$

so that $x_{1}-x_{2}=3$ so $u=\left(\begin{array}{c}1\\ -1\end{array}\right)$

take any point from that straight line : $x=\left(\begin{array}{c}4\\ 1\end{array}\right)$ $$u^Tx\neq0$$

So we conclude that points on this line (hyperplane) are not orhogonal to vector of coefficients $u$, and this is the case (they are othogonal) only if: $$H=\left\{ x\ \epsilon\ R^2 : u^Tx=0 \right\}$$

In case of plane we have following definition of hyperplane : $$H=\left\{ x\ \epsilon\ R^3 : u^Tx=v \right\}$$

then similarily $x$ and $u$ are orthogonal only in case $v=0$

I came across following definition which confused me a little bit:

$$H=\left\{ x\ \epsilon\ R^n : u^Tx=v \right\}=\left\{ x\ \epsilon\ R^n : u^T(x-a)=0 \right\}$$ where $a$ is any point on hyperplane ( so $u^Ta-v=0$). Therefore, the hyperplane H consists of the points x for which $<u, x — a> = 0$. In other words, the hyperplane H consists of the points x for which the vectors u and x-a are orthogonal

So this is the case such that $(x-a)\ \epsilon\ R^n$ and $v=0$, thats why $(x-a)$ is orthogonal to $u$.

So why in literature about for instance about Suppor Vector Machines, we assume that vector of coefficients $u$ is orthogonal to separating hyperplane, for cases where $v\neq0$? Shouldn't it be like if $u$ is orthogonal to some hyperplane, then $u$ is orthogonal to every vector $x$ in such hyperplane?

I certainly misuse some concepts, so could you provide step by step proof of the fact that vector of coefficients $u$ is orthogonal to any hyperplane?

$\endgroup$
10
  • $\begingroup$ They didn't choose $v=0$ there, they used the definition of hyperplane for the single point $a$ so that $u^T x - v = u^T x - u^T a = u^T ( x-a)=0$ $\endgroup$ – Triatticus Mar 25 '17 at 19:05
  • $\begingroup$ yes, but you can interpret this as hyperplane with v=0 and in such hyperplane coefficient vector u is orthogonal to every vector x-a in this hyperplane. $\endgroup$ – mokebe Mar 25 '17 at 19:16
  • $\begingroup$ well not really, since $v$ is simply the vector $u^T a$ and is thus not zero, it was absorbed into the left hand side of the equation instead, it is still there though. In your example you found such an $a$ when you solved for $x$. The problem in your reasoning is that the vector $\binom{4}{1}$ does not lie on the line and by itself wont be orthogonal to $u$. But find a second vector on this line, say $\binom{5}{2}$ and take their difference to obtain $\binom{1}{1}$ which is indeed orthogonal to the $u$ and lies on the line given. The condition for $v$ to be zero is either $u \perp a$ or $a=0$ $\endgroup$ – Triatticus Mar 25 '17 at 19:25
  • 1
    $\begingroup$ The exact same reasoning applies to higher dimensional objects, two vectors that point to separate points on a (hyper)plane, their difference lies in the plane and is orthogonal to the direction vector $u$ of the (hyper)plane $\endgroup$ – Triatticus Mar 25 '17 at 19:53
  • 1
    $\begingroup$ Well as for proof I don't know if I have sufficient proof for you, but that is the whole idea in terms of planes. The direction vector is orthogonal to some vector IN the plane formed by the difference between any to vectors that point to points contained in the plane...that is in fact how the definition was developed. Hopefully someone can enlighten you with a more rigorous approach $\endgroup$ – Triatticus Mar 25 '17 at 20:14
7
$\begingroup$

So to formalize discussion with Triatticus, proof is following:

If we have some hyperplane given by: $$H=\left\{ x\ \epsilon\ R^n : u^Tx=v \right\}$$

Then any two vectors $a,b$ which point to any separate points in this hyperplane are such that:

$$\begin{cases}u^Ta=v \\u^Tb=v \end{cases}$$

If we want to check whether hyperplane is orthogonal to normal vector $u$, then we have to find whether $u^T(a-b)=0$ for any a,b , because $(a-b)$ is a vector IN the hyperplane, whereas a and b are vectors which point in direction of points in the hyperplane.

If we solve above linear equations we have : $$u^T(a-b)=0$$ and this completes proof.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.