# Singular Value Decomposition (SVD) - Odd step in proof

There are a lot of proofs about the singular value decomposition of a matrix $$A \in \mathbb{R}^{m \times n}$$. Now this is the start of the proof in my textbook:

Take the symmetric matrix $$A^T \cdot A$$. Due to the spectral theorem, there exists an orthogonal matrix $$V$$ such that $$V^{-1} \cdot (A^TA)\cdot V = \Lambda$$, a diagonal matrix. The set of the columns of $$V$$ are an orthonormal basis of eigenvectors $$\{v_1, ..., v_n\}$$ in $$\mathbb{R}^n$$.

The textbook shows how $$||Av_i||^2 = \lambda_i$$, with $$\lambda_i$$ the eigenvalue of the eigenvector $$v_i$$. Now order the eigenvectors $$v_i$$ such that their corresponding eigenvalues are ordered from great to small, so that $$\lambda_1 \geq \lambda_2 \geq ... \geq \lambda_n \geq 0$$.

Now, note $$\sigma_i = \sqrt{\lambda_i} = ||Av_i||$$ for the singular values.

Suppose that there are $$r$$ singular values of $$A$$ that are non-zero: $$\sigma_1 \geq \sigma_2 \geq ... \sigma_r > 0 = \sigma_{r+1} = ... = \sigma_n$$.

Then the vectors $$A v_1, Av_2, ..., Av_r$$ are an orthogonal set of vectors, all non-zero, and so they are linear independent.

Now, my book says the following:

Since $$\{v_1, ..., v_j\}$$ is a basis for $$\mathbb{R}^n$$, the span of these vectors is the column space of A and so the image of $$L_A$$. Therefore, $$r$$ is the rank of $$A$$ and the dimension of $$L_A$$.

I frowned at this step. How can you tell that the span of $$A v_1, Av_2, ..., Av_r$$ is the column space of $$A$$? What exactly is $$Av_i$$, in fact?

(The proof continues by orthonormalizing the vectors $$Av_i$$. Then it states $$AV = \begin{pmatrix}\sigma_1Av_1 & \sigma_2Av_2 & ... & \sigma_rAv_r & 0 & 0 & ... & 0 \end{pmatrix}$$, which I understand, and by creating the matrix $$\Sigma$$ of eigenvalues. The equation $$U\Sigma V^T = A$$ is the result.)

Please tell me if this is a duplicate question, but I haven't found it yet. I read Stuck in understanding Singular Value Decomposition (SVD) , but that proof is in the other direction: the rank is assumed there.

The column space of $$A$$ is the space spanned by its collumns. But, since the $$k$$th columns is $$Ae_k$$ (where $$e_k$$ is the $$k$$th vector of the standard basis of $$\mathbb R^n$$), the columns space of $$A$$ is$$\operatorname{span}\bigl(\{Ae_1,Ae_2,\ldots,Ae_n\}\bigr).$$But, since $$\{v_1,\ldots,v_n\}$$ is another basis of $$\mathbb R^n$$,$$\operatorname{span}\bigl(\{Ae_1,Ae_2,\ldots,Ae_n\}\bigr)=\operatorname{span}\bigl(\{Av_1,Av_2,\ldots,Av_n\}\bigr).$$