orthogonal projection of a point $\vec{\text{b}}$ to the subspace wrt A

Question

A post on Quroa:

https://www.quora.com/What-is-special-about-the-matrices-AA-T-and-A-TA-Why-do-they-show-up-in-things-like-least-squares-and-SVD-I-would-like-an-intuitive-or-geometric-interpretation-of-why-these-matrices-and-their-eigenvalues-and-eigenvectors-accomplish-what-they-do#

stated that the orthogonal projection of a point $\vec{\text{b}}$ to the "subspace" defined by $A\vec{\text{x}}=\vec{\text{b}}$ is given by $$A^T(A\vec{\text{x}_0}-\vec{\text{b}})=0$$

I'm not clear how this equation can be useful since $A\vec{\text{x}_0}-\vec{\text{b}}=\vec{\text{0}}$ and any vector dotted with a zero vector will yield zero. isn't the above equation redundant?

Also, second question, is it correct that, for $\vec{\text{x}_0}$ to exist, $A$ must not be a full rank matrix? if so, how to formally prove that?

Answer 1

To my best guess, the author made a typo in his original statement. According to the purpose of his answer, the coordinate should be given by:

\begin{equation*} \begin{cases} A\vec{\text{x}}=\vec{\text{b}} \\ A^T(\vec{\text{x}}-\vec{\text{b}})=0 \end{cases} \end{equation*}

Here is my example:

\begin{equation*} A=\begin{pmatrix} 2 & 3 \\ 4 & 6 \end{pmatrix} \qquad \vec{b}=\begin{pmatrix} 7 \\ 14 \end{pmatrix} \end{equation*}

plugging them into the equation yields: \begin{equation*} \vec{x}=\begin{pmatrix} -91 \\ 63 \end{pmatrix} \end{equation*}

which is verified to be in the column space of A and $\vec{x}-\vec{b}$ is perpendicular to the column space.