Let $A \in \mathbb{R}^{m \times n}$ be of rank $r$, and $A = U\Sigma V^T$ be its SVD with $\Sigma \in \mathbb{R}^{r \times r}$. Let $P_U = UU^T$ and $P_V = VV^T$ be orthogonal projectors onto the ranges of $U$ and $V$ respectively. The operator

$$\mathcal{P}_T(Z) = P_U Z + Z P_V - P_U Z P_V$$

acting on $Z \in \mathbb{R}^{m \times n}$ is an orthogonal projector onto a subspace $T$, which is a linear space spanned by elements of the form $u_k y^T$ and $x v_k^T$ for $k = 1, \dots, r$.

Can anyone provide some further intuition about $T$? How should one think about it?

  • $\begingroup$ Since $U$ and $V$ in the real SVD are orthogonal, $UU^T$ and $VV^T$ are both the identity. While technically the identity is an orthogonal projector, I'm pretty sure it's not what you had in mind. $\endgroup$
    – celtschk
    Aug 23 '15 at 6:38
  • $\begingroup$ @celtschk Sorry, I should have clarified: We assume a reduced SVD where $\Sigma \in \mathbb{R}^{r \times r}$, $U\in \mathbb{R}^{m \times r}$ and $V \in \mathbb{R}^{n \times r}$, so $UU^T$ and $VV^T$ are not necessarily the identity. $\endgroup$
    – rodms
    Aug 23 '15 at 6:51
  • $\begingroup$ Ah, I see. But "projectors onto $U$ and $V$" doesn't make sense. Rather, they are the projectors onto the image and the orthogonal space of the kernel. $\endgroup$
    – celtschk
    Aug 23 '15 at 7:06
  • $\begingroup$ Right. I did abuse notation there, I've corrected it! $\endgroup$
    – rodms
    Aug 23 '15 at 7:10
  • $\begingroup$ Since $\mathcal P_T$ maps between two different spaces, it cannot be a projector, since the defining equation $P^2=P$ is not even defined for $P=\mathcal P_T$. $\endgroup$
    – celtschk
    Aug 23 '15 at 7:31

A better understanding of $\mathcal P_T$ can be achieved by looking at a special case: Namely when $P_U$ and $P_V$ are diagonal, with all the $1$ entries coming first. For example, consider $m=3$, $n=4$, $r=2$, and $$A = \begin{pmatrix} 1 & 2 & \color{grey}{0} & \color{grey}{0}\\ 3 & 4 & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}$$ From this, one easily can calculate that $$P_U = \begin{pmatrix} 1 & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & 1 & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}, \quad P_V = \begin{pmatrix} 1 & \color{grey}{0} & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & 1 & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} & \color{grey}{0}\\ \color{grey}{0} & \color{grey}{0} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}$$ Then for a general $3\times 4$ matrix $$Z = \begin{pmatrix} z_{11} & z_{12} & z_{13} & z_{14}\\ z_{21} & z_{22} & z_{23} & z_{24}\\ z_{31} & z_{32} & z_{33} & z_{34} \end{pmatrix}$$ you get $$\mathcal P_T(Z)= \begin{pmatrix} \color{red}{z_{11}} & \color{red}{z_{12}} & z_{13} & z_{14}\\ \color{red}{z_{21}} & \color{red}{z_{22}} & z_{23} & z_{24}\\ z_{31} & z_{32} & \color{grey}{0} & \color{grey}{0} \end{pmatrix}$$ So you can see that for $v\in\ker(A)$, $\mathcal P_T(Z)v$ = $P_UZv$, that is, after applying $Z$, they are projected to the range of $A$. However, for vectors orthogonal to the kernel, the image under $P_T(Z)$ is just the image under $Z$. That is, $\mathcal P_T$ removes that part of $Z$ that maps the kernel of $A$ to the space orthogonal to the range of $A$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.