# Intuition about an orthogonal projection operator for matrices

Let $$A \in \mathbb{R}^{m \times n}$$ be of rank $$r$$, and $$A = U\Sigma V^T$$ be its SVD with $$\Sigma \in \mathbb{R}^{r \times r}$$. Let $$P_U = UU^T$$ and $$P_V = VV^T$$ be orthogonal projectors onto the ranges of $$U$$ and $$V$$ respectively. The operator

$$\mathcal{P}_T(Z) = P_U Z + Z P_V - P_U Z P_V$$

acting on $$Z \in \mathbb{R}^{m \times n}$$ is an orthogonal projector onto a subspace $$T$$, which is a linear space spanned by elements of the form $$u_k y^T$$ and $$x v_k^T$$ for $$k = 1, \dots, r$$.

Can anyone provide some further intuition about $$T$$? How should one think about it?

• Since $U$ and $V$ in the real SVD are orthogonal, $UU^T$ and $VV^T$ are both the identity. While technically the identity is an orthogonal projector, I'm pretty sure it's not what you had in mind. Aug 23 '15 at 6:38
• @celtschk Sorry, I should have clarified: We assume a reduced SVD where $\Sigma \in \mathbb{R}^{r \times r}$, $U\in \mathbb{R}^{m \times r}$ and $V \in \mathbb{R}^{n \times r}$, so $UU^T$ and $VV^T$ are not necessarily the identity. Aug 23 '15 at 6:51
• Ah, I see. But "projectors onto $U$ and $V$" doesn't make sense. Rather, they are the projectors onto the image and the orthogonal space of the kernel. Aug 23 '15 at 7:06
• Right. I did abuse notation there, I've corrected it! Aug 23 '15 at 7:10
• Since $\mathcal P_T$ maps between two different spaces, it cannot be a projector, since the defining equation $P^2=P$ is not even defined for $P=\mathcal P_T$. Aug 23 '15 at 7:31

A better understanding of $\mathcal P_T$ can be achieved by looking at a special case: Namely when $P_U$ and $P_V$ are diagonal, with all the $1$ entries coming first. For example, consider $m=3$, $n=4$, $r=2$, and $$A = \begin{pmatrix} 1 & 2 & \color{grey}{0} & \color{grey}{0}\\ 3 & 4 & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}$$ From this, one easily can calculate that $$P_U = \begin{pmatrix} 1 & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & 1 & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}, \quad P_V = \begin{pmatrix} 1 & \color{grey}{0} & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & 1 & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}$$ Then for a general $3\times 4$ matrix $$Z = \begin{pmatrix} z_{11} & z_{12} & z_{13} & z_{14}\\ z_{21} & z_{22} & z_{23} & z_{24}\\ z_{31} & z_{32} & z_{33} & z_{34} \end{pmatrix}$$ you get $$\mathcal P_T(Z)= \begin{pmatrix} \color{red}{z_{11}} & \color{red}{z_{12}} & z_{13} & z_{14}\\ \color{red}{z_{21}} & \color{red}{z_{22}} & z_{23} & z_{24}\\ z_{31} & z_{32} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}$$ So you can see that for $v\in\ker(A)$, $\mathcal P_T(Z)v$ = $P_UZv$, that is, after applying $Z$, they are projected to the range of $A$. However, for vectors orthogonal to the kernel, the image under $P_T(Z)$ is just the image under $Z$. That is, $\mathcal P_T$ removes that part of $Z$ that maps the kernel of $A$ to the space orthogonal to the range of $A$.