There are a lot of proofs about the singular value decomposition of a matrix $A \in \mathbb{R}^{m \times n}$. Now this is the start of the proof in my textbook:
Take the symmetric matrix $A^T \cdot A$. Due to the spectral theorem, there exists an orthogonal matrix $V$ such that $V^{-1} \cdot (A^TA)\cdot V = \Lambda$, a diagonal matrix. The set of the columns of $V$ are an orthonormal basis of eigenvectors $\{v_1, ..., v_n\}$ in $\mathbb{R}^n$.
The textbook shows how $||Av_i||^2 = \lambda_i$, with $\lambda_i$ the eigenvalue of the eigenvector $v_i$. Now order the eigenvectors $v_i$ such that their corresponding eigenvalues are ordered from great to small, so that $\lambda_1 \geq \lambda_2 \geq ... \geq \lambda_n \geq 0$.
Now, note $\sigma_i = \sqrt{\lambda_i} = ||Av_i||$ for the singular values.
Suppose that there are $r$ singular values of $A$ that are non-zero: $\sigma_1 \geq \sigma_2 \geq ... \sigma_r > 0 = \sigma_{r+1} = ... = \sigma_n$.
Then the vectors $A v_1, Av_2, ..., Av_r$ are an orthogonal set of vectors, all non-zero, and so they are linear independent.
Now, my book says the following:
Since $\{v_1, ..., v_j\}$ is a basis for $\mathbb{R}^n$, the span of these vectors is the column space of A and so the image of $L_A$. Therefore, $r$ is the rank of $A$ and the dimension of $L_A$.
I frowned at this step. How can you tell that the span of $A v_1, Av_2, ..., Av_r$ is the column space of $A$? What exactly is $Av_i$, in fact?
(The proof continues by orthonormalizing the vectors $Av_i$. Then it states $AV = \begin{pmatrix}\sigma_1Av_1 & \sigma_2Av_2 & ... & \sigma_rAv_r & 0 & 0 & ... & 0 \end{pmatrix}$, which I understand, and by creating the matrix $\Sigma$ of eigenvalues. The equation $U\Sigma V^T = A$ is the result.)
Please tell me if this is a duplicate question, but I haven't found it yet. I read Stuck in understanding Singular Value Decomposition (SVD) , but that proof is in the other direction: the rank is assumed there.
