# Orthogonal Projection

Seems like I still don't get it, I think I am missing something important.

Let $V$ be an $n$ dimensional inner product space ($n \geq 1$), and $T\colon\mathbf{V}\to\mathbf{V}$ be a linear transformation such that:

• $T^2 = T$
• $||T(a)|| \leq ||a||$ for every vector $a$ in $\mathbf{V}$;

Prove that a subspace $U \subseteq V$ exist, such that $T$ is the orthogonal projection on $U$.

Now, I know these things:

• The fact hat $T^2 = T$ guarantees that $T$ is indeed a projection, so I need to prove that T is an orthogonal projection (I guess this is where $||T(a)|| \leq ||a||$ kicks in).
• To do this I can prove that:
• For every $v$ in $ImT^{\perp}$, $T(v) = 0$
• Alternatively, I can prove that for every $v$ in $ImT$ and $u$ in $KerT$, $(v,u)=0$.
• $T$ is self-adjoint (according to Wikipedia)
• The matrix $A = [T]_{E}$ when $E$ is an orthonormal basis, is hermitian (this is equivalent to the previous point).
• What else?

I've been thinking about it for quite some time now, and I'm pretty sure there is something big I'm missing, again. I just don't know how to use the data to prove any of these things.

Thanks!

• Rule of thumb: if you don't know how to prove something, try to find a counterexample. Eventually you will say to yourself "well, of course I can't find a counterexample, because (blank) is stopping me," and you can probably turn (blank) into a proof. Feb 12, 2011 at 15:13

One approach to this is by using Pythagorean Theorem. You fill in the details.

Indeed, suppose that $T \psi \in Im T$. Note that $V = Ker T \oplus Ker T^\bot$. Now write $T \psi = k + k'$ where $k \in Ker T$ and $k' \in Ker T^\bot$. We get that

$$\|k'\|^2 \ge \|T k'\|^2 = \|T k + T k'\|^2 = \|T(T \psi)\|^2 = \|T \psi\|^2 = \|k\|^2 + \|k'^2\|,$$

from which we gather that $k = 0$. Therefore $T \psi \in Ker T^\bot$. Thus $Im T \subset Ker T^\bot$. On the other hand if $\phi \in Im T^\bot \cap Ker T^\bot$, then one can show that $\phi \in Im T^* \cap Ker T^*$. (Because $Im T^\bot = Ker T^*$.) One can also show that $T^*$ is a projection. Let $\phi = T^* \psi$. Then $T^* \psi = T^* T^* \psi = T^* \phi = 0$. Hence $\phi = 0$. Since $V = Im T \oplus Im T^\bot$, we get that $Ker T^\bot \subset Im T$.

• Thanks! May I ask how you came up with the first part? It's easy for me to think about things like the second part of your proof (because I just "visualize" it), but what you did with the theorem just seems like symbol manipulation to me. I mean, I can see why it is working, now that it is laid before me, but I wouldn't know how to get there if I had to.
– Hila
Feb 12, 2011 at 21:36
• Hila: Qiaochu's advice is sound and drawing a picture helps, too. If you draw a non-orthogonal projection onto a line in the plane along with its Im, Ker and Ker^\bot, you'll see quite quickly what's going to go wrong with the second condition you have. Your intuition here should be that if the projection is non-orthogonal then there will be a vector whose length gets increased in the mapping. Taking the direct sum with Ker T and Ker^\bot is also a quite natural thing to try, since that will allow you to concentrate separately on the part of the vector which is significant in the projection. Feb 13, 2011 at 7:36
• ... continued ... And of course taking that direct sum lets you to use Pythagorean theorem as well, which is a good tool for calculating lengths of vectors. Feb 13, 2011 at 7:36

Here is an approach which doesn't use the decomposition of $$V$$ into a subspace and its orthogonal complement, nor finite dimension for that matter. However, I'll use the following characterization of orthogonal projection: A linear map $$T\colon V\to V$$ such that $$T^2=T$$ and such that $$\langle v-T(v),T(w)\rangle=0$$ for all $$v,w$$.

I'll take the complex case as there are some extra details, but the real case is analogous.

So we use the inequality $$\Vert T(a)\Vert^2\leq\Vert a\Vert^2$$ with $$a=v+T(t w)$$, where $$v,w\in V$$ and $$t>0$$: \begin{align*}\Vert T(v+T(tw))\Vert^2&\leq\Vert v+T(t w)\Vert^2\\\Vert T(v)\Vert^2+2\operatorname{Re}\langle T(v),T(t w)\rangle+\Vert T(t w)\Vert^2&\leq\Vert v\Vert^2+2\operatorname{Re}\langle v,T(t w)\rangle+\Vert T(t w)\Vert^2\\\Vert T(v)\Vert^2+2\operatorname{Re}\langle T(v),T(t w)\rangle&\leq\Vert v\Vert^2+2\operatorname{Re}\langle v,T(t w)\rangle\\t^{-1}\Vert T(v)\Vert^2+2\operatorname{Re}\langle T(v),T(w)\rangle&\leq t^{-1}\Vert v\Vert^2+2\operatorname{Re}\langle v,T(w)\rangle,\end{align*} where the last line is obtained from the previous one because $$t>0$$. Letting $$t\to\infty$$ we obtain \begin{align*}2\operatorname{Re}\langle T(v),T(w)\rangle\leq 2\operatorname{Re}\langle v,T(w)\rangle\end{align*} for all $$v,w$$. Simplifying the "$$2$$" term and using the same inequality with $$-v$$ in place of $$v$$ yields $$\operatorname{Re}\langle T(v),T(w)\rangle=\operatorname{Re}\langle v,T(w)\rangle$$ for all $$v,w$$. For a complex number $$z$$, we have $$\operatorname{Im}(z)=\operatorname{Re}(-iz)$$, so using the equality above with $$iw$$ in place of $$w$$ yields $$\operatorname{Im}\langle T(v),T(w)\rangle=\operatorname{Im}\langle v,T(w)\rangle$$ for all $$v,w$$. So the last two equalities give us $$\langle T(v),T(w)\rangle=\langle v,T(w)\rangle$$ for all $$v,w$$, and this means that $$T$$ is an orthogonal projection (onto $$T(V)$$), according to the definition above.

A different proof in the finite-dimensional case:

Using Schur triangularization there exists an orthonormal basis $$\{v_1, \ldots, v_n\}$$ for $$V$$ such that the matrix of $$T$$ is upper triangular: $$\begin{bmatrix} t_{11} & t_{12} & t_{13} & \cdots & t_{1n}\\ 0 & t_{22} & t_{23} & \cdots & t_{2n} \\ 0 & 0 & t_{33} & \cdots & t_{2n} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & t_{nn} \end{bmatrix}$$ We wish to prove that the matrix is diagonal. From $$T^2 = T$$ it follows directly that $$t_{ii} \in \{0,1\}$$ for all $$1\le i\le n$$.

Assume that for some $$1 \le k \le n$$ we have $$t_{ij} = 0$$ for all $$1 \le j \le k-1$$ and $$1 \le i \le j-1$$.

If $$t_{kk} = 1$$ then $$\sum_{i=1}^{k-1}|t_{ik}|^2+1=\sum_{i=1}^{k}|t_{ik}|^2 = \left\|\sum_{i=1}^{k} t_{ik}v_i\right\|^2 = \|Tv_k\|^2 \le \|v_k\|^2 = 1$$ so $$t_{1k} = t_{2k} = \cdots = t_{(k-1)k} = 0$$.

If $$t_{kk} = 0$$, notice that $$t_{ik} \ne 0$$ for some $$1 \le i \le k-1$$ implies $$t_{ii} = 0$$. Indeed, we have $$T(v_i+\overline{t_{ik}}v_k) = \sum_{r=1}^{k-1}\overline{t_{ik}}t_{rk}v_r + t_{ii} v_i = \overline{t_{ik}}\sum_{r=1\\ r \ne k}^{k-1}t_{rk}v_r + (t_{ii}+|t_{ik}|^2)v_i$$ and hence $$t_{ii}+|t_{ik}|^2 = \left|\left\langle T(v_i+\overline{t_{ik}}v_k), v_i\right\rangle\right|\le \|T(v_i+\overline{t_{ik}}v_k)\| \le \|v_i+\overline{t_{ik}}v_k\| = \sqrt{1+|t_{ik}|^2}$$ so it cannot be $$t_{ii}=1$$.

Now we have $$Tv_k = T^2v_k = \sum_{i=1}^{k-1}t_{ik}Tv_i=\sum_{i=1}^{k-1}t_{ik}t_{ii}v_i = 0$$ since $$t_{ik} \ne 0$$ implies $$t_{ii} = 0$$. In particular $$t_{1k} = t_{2k} = \cdots = t_{(k-1)k} = 0$$.

Since the base case is vacuously true, by induction we conclude that the matrix is diagonal. Therefore $$T$$ can be represented by a diagonal matrix with only $$0$$ and $$1$$ on the diagonal, so $$T$$ is an orthogonal projection.