0
$\begingroup$

This is exercise 4.2.12 of Fundamentals of matrix Computations from Watkins:

Let $A \in\mathbb R^{n\times m}$, $n \geq m$, $\operatorname{rank}(A) = m$ with complete SVD $ A = U\Sigma V^T$, $U\in \mathbb R^{n\times n}$ and $V\in R^{m\times m}$ are orthogonal matrices and $\Sigma\in \mathbb R^{n\times m} = \operatorname{diag}(\sigma_1,\ldots,\sigma_m)$ with singular values $\sigma_1\geq \sigma_2\geq\cdots\geq \sigma_m\geq 0 $.

I need to find the SVD for $A(A^T A)^{-1} A^T$ :

$$A(A^T A)^{-1} A^T = (U \Sigma V^T)(V(\Sigma^T \Sigma)^{-1} V^T)(V \Sigma^T U^T) = U \Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T U^T. $$

Since $\Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T = \operatorname{diag}(\frac{1}{\sigma_1},\ldots,\frac{1}{\sigma_m}) \operatorname{diag}(\sigma_1,\ldots,\sigma_m) = I_n$ , we have that $ A(A^T A)^{-1} A^T = U I_n U^T = I_n$ . But I think I should obtain $V^T$ in the last expression instead of $U^T$.

What did I do wrong ? Any help will be appreciated

$\endgroup$
1
  • 2
    $\begingroup$ In general, $\Sigma (\Sigma^T \Sigma)^{-1}\Sigma^T$ won't equal the identity matrix. In fact, $\Sigma (\Sigma^T \Sigma)^{-1}\Sigma^T$ is the matrix for projection onto the column space of $\Sigma$, which equals the identity iff the column space of $\Sigma$ is $\Bbb{R}^{n}$ (which occurs iff $\mathrm{rank(A)} = n$ here). $\endgroup$ Commented May 13, 2020 at 18:25

2 Answers 2

2
$\begingroup$

The mistake is that $\Sigma (\Sigma^T\Sigma)^{-1} \Sigma^T \neq I_{n}$. Note that $\Sigma = \begin{bmatrix} \operatorname{diag}(\sigma_i)_{m \times m} \\ 0_{n-m \times m} \end{bmatrix}$. Then $$\Sigma (\Sigma^{T}\Sigma)^{-1} \Sigma^{T}=\begin{bmatrix} I_{m \times m} & 0 \\ 0 & 0_{n-m \times n-m} \end{bmatrix}$$

$\endgroup$
2
$\begingroup$

$\Sigma\in\mathbb R^{n\times m} = \operatorname{diag}(\sigma_1,\ldots,\sigma_m)$

This is wrong. $\Sigma$ is a rectangular matrix, not a square matrix.

Since $\Sigma(\Sigma^T \Sigma)^{-1} \Sigma^T = \operatorname{diag}(\frac{1}{\sigma_1},\ldots,\frac{1}{\sigma_m}) \operatorname{diag}(\sigma_1,\ldots,\sigma_m) = I_n$

This is also wrong. Since $\min(n,m)=m$, the rank of $\Sigma$ is at most $m$. So is the rank of $\Sigma(\Sigma^T \Sigma)^{-1}\Sigma^T$. Hence $\Sigma(\Sigma^T \Sigma)^{-1}\Sigma^T$ cannot possibly be equal to $I_n$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .