There are a lot of proofs about the singular value decomposition of a matrix $A \in \mathbb{R}^{m \times n}$. Now this is the start of the proof in my textbook:

Take the symmetric matrix $A^T \cdot A$. Due to the spectral theorem, there exists an orthogonal matrix $V$ such that $V^{-1} \cdot (A^TA)\cdot V = \Lambda$, a diagonal matrix. The set of the columns of $V$ are an orthonormal basis of eigenvectors $\{v_1, ..., v_n\}$ in $\mathbb{R}^n$.

The textbook shows how $||Av_i||^2 = \lambda_i$, with $\lambda_i$ the eigenvalue of the eigenvector $v_i$. Now order the eigenvectors $v_i$ such that their corresponding eigenvalues are ordered from great to small, so that $\lambda_1 \geq \lambda_2 \geq ... \geq \lambda_n \geq 0$.

Now, note $\sigma_i = \sqrt{\lambda_i} = ||Av_i||$ for the singular values.

Suppose that there are $r$ singular values of $A$ that are non-zero: $\sigma_1 \geq \sigma_2 \geq ... \sigma_r > 0 = \sigma_{r+1} = ... = \sigma_n$.

Then the vectors $A v_1, Av_2, ..., Av_r$ are an orthogonal set of vectors, all non-zero, and so they are linear independent.

Now, my book says the following:

Since $\{v_1, ..., v_j\}$ is a basis for $\mathbb{R}^n$, the span of these vectors is the column space of A and so the image of $L_A$. Therefore, $r$ is the rank of $A$ and the dimension of $L_A$.

I frowned at this step. How can you tell that the span of $A v_1, Av_2, ..., Av_r$ is the column space of $A$? What exactly is $Av_i$, in fact?

(The proof continues by orthonormalizing the vectors $Av_i$. Then it states $AV = \begin{pmatrix}\sigma_1Av_1 & \sigma_2Av_2 & ... & \sigma_rAv_r & 0 & 0 & ... & 0 \end{pmatrix}$, which I understand, and by creating the matrix $\Sigma$ of eigenvalues. The equation $U\Sigma V^T = A$ is the result.)

Please tell me if this is a duplicate question, but I haven't found it yet. I read Stuck in understanding Singular Value Decomposition (SVD) , but that proof is in the other direction: the rank is assumed there.


1 Answer 1


The column space of $A$ is the space spanned by its collumns. But, since the $k$th columns is $Ae_k$ (where $e_k$ is the $k$th vector of the standard basis of $\mathbb R^n$), the columns space of $A$ is$$\operatorname{span}\bigl(\{Ae_1,Ae_2,\ldots,Ae_n\}\bigr).$$But, since $\{v_1,\ldots,v_n\}$ is another basis of $\mathbb R^n$,$$\operatorname{span}\bigl(\{Ae_1,Ae_2,\ldots,Ae_n\}\bigr)=\operatorname{span}\bigl(\{Av_1,Av_2,\ldots,Av_n\}\bigr).$$

  • $\begingroup$ Didn't know that it would be that simple! Thank you very much, Sir. $\endgroup$ Aug 24, 2019 at 16:09
  • $\begingroup$ I'm glad I could help. $\endgroup$ Aug 24, 2019 at 16:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .