There are a lot of proofs about the singular value decomposition of a matrix $A \in \mathbb{R}^{m \times n}$. Now this is the start of the proof in my textbook:

Take the symmetric matrix $A^T \cdot A$. Due to the spectral theorem, there exists an orthogonal matrix $V$ such that $V^{-1} \cdot (A^TA)\cdot V = \Lambda$, a diagonal matrix. The set of the columns of $V$ are an orthonormal basis of eigenvectors $\{v_1, ..., v_n\}$ in $\mathbb{R}^n$.

The textbook shows how $||Av_i||^2 = \lambda_i$, with $\lambda_i$ the eigenvalue of the eigenvector $v_i$. Now order the eigenvectors $v_i$ such that their corresponding eigenvalues are ordered from great to small, so that $\lambda_1 \geq \lambda_2 \geq ... \geq \lambda_n \geq 0$.

Now, note $\sigma_i = \sqrt{\lambda_i} = ||Av_i||$ for the singular values.

Suppose that there are $r$ singular values of $A$ that are non-zero: $\sigma_1 \geq \sigma_2 \geq ... \sigma_r > 0 = \sigma_{r+1} = ... = \sigma_n$.

Then the vectors $A v_1, Av_2, ..., Av_r$ are an orthogonal set of vectors, all non-zero, and so they are linear independent.

Now, my book says the following:

Since $\{v_1, ..., v_j\}$ is a basis for $\mathbb{R}^n$, the span of these vectors is the column space of A and so the image of $L_A$. Therefore, $r$ is the rank of $A$ and the dimension of $L_A$.

I frowned at this step. How can you tell that the span of $A v_1, Av_2, ..., Av_r$ is the column space of $A$? What exactly is $Av_i$, in fact?

(The proof continues by orthonormalizing the vectors $Av_i$. Then it states $AV = \begin{pmatrix}\sigma_1Av_1 & \sigma_2Av_2 & ... & \sigma_rAv_r & 0 & 0 & ... & 0 \end{pmatrix}$, which I understand, and by creating the matrix $\Sigma$ of eigenvalues. The equation $U\Sigma V^T = A$ is the result.)

Please tell me if this is a duplicate question, but I haven't found it yet. I read Stuck in understanding Singular Value Decomposition (SVD) , but that proof is in the other direction: the rank is assumed there.


The column space of $A$ is the space spanned by its collumns. But, since the $k$th columns is $Ae_k$ (where $e_k$ is the $k$th vector of the standard basis of $\mathbb R^n$), the columns space of $A$ is$$\operatorname{span}\bigl(\{Ae_1,Ae_2,\ldots,Ae_n\}\bigr).$$But, since $\{v_1,\ldots,v_n\}$ is another basis of $\mathbb R^n$,$$\operatorname{span}\bigl(\{Ae_1,Ae_2,\ldots,Ae_n\}\bigr)=\operatorname{span}\bigl(\{Av_1,Av_2,\ldots,Av_n\}\bigr).$$

| cite | improve this answer | |
  • $\begingroup$ Didn't know that it would be that simple! Thank you very much, Sir. $\endgroup$ – Victor Neverland Aug 24 '19 at 16:09
  • $\begingroup$ I'm glad I could help. $\endgroup$ – José Carlos Santos Aug 24 '19 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.