# orthogonal projection of a point $\vec{\text{b}}$ to the subspace wrt A

A post on Quroa:

https://www.quora.com/What-is-special-about-the-matrices-AA-T-and-A-TA-Why-do-they-show-up-in-things-like-least-squares-and-SVD-I-would-like-an-intuitive-or-geometric-interpretation-of-why-these-matrices-and-their-eigenvalues-and-eigenvectors-accomplish-what-they-do#

stated that the orthogonal projection of a point $$\vec{\text{b}}$$ to the "subspace" defined by $$A\vec{\text{x}}=\vec{\text{b}}$$ is given by $$A^T(A\vec{\text{x}_0}-\vec{\text{b}})=0$$

I'm not clear how this equation can be useful since $$A\vec{\text{x}_0}-\vec{\text{b}}=\vec{\text{0}}$$ and any vector dotted with a zero vector will yield zero. isn't the above equation redundant?

Also, second question, is it correct that, for $$\vec{\text{x}_0}$$ to exist, $$A$$ must not be a full rank matrix? if so, how to formally prove that?

• Do you mean the subspace defined by $A\vec{x} = \vec{0}$? – Trevor Gunn Mar 18 '20 at 22:19
• Also, why is $A\vec{\text{x}_0}-\vec{\text{b}}=\vec{\text{0}}$? What assumptions are you making about $\vec{x}_0$ and $\vec b$? Are you assuming that $\vec{b}$ is in the column space of $A$? – Trevor Gunn Mar 18 '20 at 22:20
• Yeah, I think that definitely will make more sense. by the way, $A\vec{\text{x}_0}-b\vec{\text{b}_0}=\vec{\text{0}}$ because $\vec{\text{x}_0}$ is on V. – techie11 Mar 18 '20 at 22:44
• I saw this on Quora: quora.com/… – techie11 Mar 18 '20 at 22:47
• Well usually you project onto the column space of $A$ not its nullspace. So if $A\vec{x_0} = \vec b$ then $\vec b$ is already in the column space and there's nothing to do: the projection is just $\vec b$. – Trevor Gunn Mar 18 '20 at 22:54

## 1 Answer

To my best guess, the author made a typo in his original statement. According to the purpose of his answer, the coordinate should be given by:

$$\begin{equation*} \begin{cases} A\vec{\text{x}}=\vec{\text{b}} \\ A^T(\vec{\text{x}}-\vec{\text{b}})=0 \end{cases} \end{equation*}$$

Here is my example:

$$\begin{equation*} A=\begin{pmatrix} 2 & 3 \\ 4 & 6 \end{pmatrix} \qquad \vec{b}=\begin{pmatrix} 7 \\ 14 \end{pmatrix} \end{equation*}$$

plugging them into the equation yields: $$\begin{equation*} \vec{x}=\begin{pmatrix} -91 \\ 63 \end{pmatrix} \end{equation*}$$

which is verified to be in the column space of A and $$\vec{x}-\vec{b}$$ is perpendicular to the column space.