Find SVD of $A (A^T A)^{-1} A^T$

Question

This is exercise 4.2.12 of Fundamentals of matrix Computations from Watkins:

Let $A \in\mathbb R^{n\times m}$, $n \geq m$, $\operatorname{rank}(A) = m$ with complete SVD $ A = U\Sigma V^T$, $U\in \mathbb R^{n\times n}$ and $V\in R^{m\times m}$ are orthogonal matrices and $\Sigma\in \mathbb R^{n\times m} = \operatorname{diag}(\sigma_1,\ldots,\sigma_m)$ with singular values $\sigma_1\geq \sigma_2\geq\cdots\geq \sigma_m\geq 0 $.

I need to find the SVD for $A(A^T A)^{-1} A^T$ :

$$A(A^T A)^{-1} A^T = (U \Sigma V^T)(V(\Sigma^T \Sigma)^{-1} V^T)(V \Sigma^T U^T) = U \Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T U^T. $$

Since $\Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T = \operatorname{diag}(\frac{1}{\sigma_1},\ldots,\frac{1}{\sigma_m}) \operatorname{diag}(\sigma_1,\ldots,\sigma_m) = I_n$ , we have that $ A(A^T A)^{-1} A^T = U I_n U^T = I_n$ . But I think I should obtain $V^T$ in the last expression instead of $U^T$.

What did I do wrong ? Any help will be appreciated

Answer 1

The mistake is that $\Sigma (\Sigma^T\Sigma)^{-1} \Sigma^T \neq I_{n}$. Note that $\Sigma = \begin{bmatrix} \operatorname{diag}(\sigma_i)_{m \times m} \\ 0_{n-m \times m} \end{bmatrix}$. Then $$\Sigma (\Sigma^{T}\Sigma)^{-1} \Sigma^{T}=\begin{bmatrix} I_{m \times m} & 0 \\ 0 & 0_{n-m \times n-m} \end{bmatrix}$$

Answer 2

$\Sigma\in\mathbb R^{n\times m} = \operatorname{diag}(\sigma_1,\ldots,\sigma_m)$

This is wrong. $\Sigma$ is a rectangular matrix, not a square matrix.

Since $\Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T = \operatorname{diag}(\frac{1}{\sigma_1},\ldots,\frac{1}{\sigma_m}) \operatorname{diag}(\sigma_1,\ldots,\sigma_m) = I_n$

This is also wrong. Since $\min(n,m)=m$, the rank of $\Sigma$ is at most $m$. So is the rank of $\Sigma(\Sigma^T \Sigma)^{-1}\Sigma^T$. Hence $\Sigma(\Sigma^T \Sigma)^{-1}\Sigma^T$ cannot possibly be equal to $I_n$.