# Find SVD of $A (A^T A)^{-1} A^T$

This is exercise 4.2.12 of Fundamentals of matrix Computations from Watkins:

Let $$A \in\mathbb R^{n\times m}$$, $$n \geq m$$, $$\operatorname{rank}(A) = m$$ with complete SVD $$A = U\Sigma V^T$$, $$U\in \mathbb R^{n\times n}$$ and $$V\in R^{m\times m}$$ are orthogonal matrices and $$\Sigma\in \mathbb R^{n\times m} = \operatorname{diag}(\sigma_1,\ldots,\sigma_m)$$ with singular values $$\sigma_1\geq \sigma_2\geq\cdots\geq \sigma_m\geq 0$$.

I need to find the SVD for $$A(A^T A)^{-1} A^T$$ :

$$A(A^T A)^{-1} A^T = (U \Sigma V^T)(V(\Sigma^T \Sigma)^{-1} V^T)(V \Sigma^T U^T) = U \Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T U^T.$$

Since $$\Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T = \operatorname{diag}(\frac{1}{\sigma_1},\ldots,\frac{1}{\sigma_m}) \operatorname{diag}(\sigma_1,\ldots,\sigma_m) = I_n$$ , we have that $$A(A^T A)^{-1} A^T = U I_n U^T = I_n$$ . But I think I should obtain $$V^T$$ in the last expression instead of $$U^T$$.

What did I do wrong ? Any help will be appreciated

• In general, $\Sigma (\Sigma^T \Sigma)^{-1}\Sigma^T$ won't equal the identity matrix. In fact, $\Sigma (\Sigma^T \Sigma)^{-1}\Sigma^T$ is the matrix for projection onto the column space of $\Sigma$, which equals the identity iff the column space of $\Sigma$ is $\Bbb{R}^{n}$ (which occurs iff $\mathrm{rank(A)} = n$ here). Commented May 13, 2020 at 18:25

The mistake is that $$\Sigma (\Sigma^T\Sigma)^{-1} \Sigma^T \neq I_{n}$$. Note that $$\Sigma = \begin{bmatrix} \operatorname{diag}(\sigma_i)_{m \times m} \\ 0_{n-m \times m} \end{bmatrix}$$. Then $$\Sigma (\Sigma^{T}\Sigma)^{-1} \Sigma^{T}=\begin{bmatrix} I_{m \times m} & 0 \\ 0 & 0_{n-m \times n-m} \end{bmatrix}$$
$$\Sigma\in\mathbb R^{n\times m} = \operatorname{diag}(\sigma_1,\ldots,\sigma_m)$$
This is wrong. $$\Sigma$$ is a rectangular matrix, not a square matrix.
Since $$\Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T = \operatorname{diag}(\frac{1}{\sigma_1},\ldots,\frac{1}{\sigma_m}) \operatorname{diag}(\sigma_1,\ldots,\sigma_m) = I_n$$
This is also wrong. Since $$\min(n,m)=m$$, the rank of $$\Sigma$$ is at most $$m$$. So is the rank of $$\Sigma(\Sigma^T \Sigma)^{-1}\Sigma^T$$. Hence $$\Sigma(\Sigma^T \Sigma)^{-1}\Sigma^T$$ cannot possibly be equal to $$I_n$$.