2
$\begingroup$

There is a theorem that says if $U$ is an orthogonal matrix, i.e., its columns (or rows) form an orthonormal basis, then the action of $UU^T$ represents orthogonal projection of the vector space onto the space spanned by the columns of $U$.

But, why does this theorem also apply when my "$U$" matrix is a $3\times1$ column vector and hence not an orthogonal matrix (it's not even square)?

I had to compute the matrix that represents orthogonal projection onto the line spanned by $(1, 2, -1)$. So, following the Gram-Schmidt process for this simple case, just set $v_1= (1,2,-1)$. Then I normalized this vector to make it of unit length. Finally, I compute $VV^T$ to get a $3\times3$ matrix. I applied the theorem stated above and concluded that this $3\times3$ matrix represents orthogonal projection onto the line spanned by the one column in $V$, which has the same span as the span of $(1, 2,-1)$, since the Gram-Schmidt process produces a set of orthogonal vectors with the same span as the original set of linearly independent vectors.

Have I applied the theorem incorrectly? I got the correct matrix.

Thanks,

$\endgroup$
2
$\begingroup$

The theorem you have quoted is true but only tells part of the story. An improved version is as follows.

Let $U$ be a real $m\times n$ matrix with orthonormal columns, that is, its columns form an orthonormal basis of some subspace $W$ of ${\Bbb R}^m$. Then $UU^T$ is the matrix of the projection of ${\Bbb R}^m$ onto $W$.

Comments

  • The restriction to real matrices is not actually necessary, any scalar field will do, and any vector space, just so long as you know what "orthonormal" means in that vector space.
  • A matrix with orthonormal columns is an orthogonal matrix if it is square. I think this is the situation you are envisaging in your question. But in this case the result is trivial because $W$ is equal to ${\Bbb R}^m$, and $UU^T=I$, and the projection transformation is simply $P({\bf x})={\bf x}$.
$\endgroup$
  • $\begingroup$ Awesome answer, @David. Thanks so much for the clarification and for the helpful comments. Just one thing: I think, in the box highlighted in yellow in your answer, you might want to say explicitly that $UU^T$ represents orthogonal projection onto W. $\endgroup$ – user247597 Jun 12 '15 at 2:33
  • $\begingroup$ For me, "projection" means "orthogonal projection". However, no objection if you want to include the adjective ;-) $\endgroup$ – David Jun 12 '15 at 2:35
  • $\begingroup$ Haha...ok, got it. Thanks @David :-) $\endgroup$ – user247597 Jun 12 '15 at 2:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.