Suppose $Rank(A)=r,A{\in}R^{m \times n}$, $v_1,v_2,...,v_r{\in}R^n$ forms an orthonormal basis of $C(A^T)$(i.e. row space of A) and are eigenvectors of $A^TA$.

Now, take $\sigma_1u_1=Av_1,\sigma_2u_2=Av_2,...,\sigma_ru_r=Av_r;\ \sigma_1,\sigma_2,...,\sigma_r$ are positive scaling factors and $u_1,u_2,...,u_r{\in}R^m$ are unit vectors in $C(A)$(column space of A).

Now,we have $A(v_1,v_2,...,v_r)=(u_1,u_2,...,u_r)\begin{bmatrix}\sigma_1& 0 &\cdots &0 \\ 0& \sigma_2 & \cdots &0 \\ \vdots &\vdots &\ddots &\vdots \\0&0&\cdots&\sigma_r\end{bmatrix}$. And denote this formula as $AV=U\Sigma,V{\in}R^{n\times r},U{\in}R^{m\times r},\Sigma$ is a r$\times$r diagonal matrix.

My problem is how to show $u_1,u_2,...,u_r$ is an orthonormal basis of $C(A)$? E.g. $u_1^Tu_2=0$ when take $\sigma_1u_1=Av_1,\sigma_2u_2=Av_2$.

Above thread comes from MIT's opencourse 18.06 linear algebra lecture 29

  • $\begingroup$ The vectors $v_i$ must be chosen in a special way. $\endgroup$ – littleO Aug 30 '17 at 2:15
  • $\begingroup$ @littleO : Can you state the trick concretely? $\endgroup$ – Finley Aug 30 '17 at 2:32
  • $\begingroup$ @Finley This is a very strange way to present the idea of finding an SVD. Where is this question coming from; that is, what have you learned/read most recently? Have you tried reading a textbook and finding the SVD with the method outline there? $\endgroup$ – Omnomnomnom Aug 30 '17 at 2:39
  • $\begingroup$ @Finley how about this: are you at least aware that the $\sigma_i$ must be the square roots of the eigenvalues of $A^TA$? $\endgroup$ – Omnomnomnom Aug 30 '17 at 2:42
  • $\begingroup$ Here's a relevant thread that explains one way of understanding the SVD: math.stackexchange.com/questions/1737637/… $\endgroup$ – littleO Aug 30 '17 at 2:42

Assuming everything in the problem statement, ... \begin{align*} u_i^\mathrm{T} u_j &= \frac{1}{\sigma_i \sigma_j} \sigma_i \sigma_j u_i^\mathrm{T} u_j \\ &= \frac{1}{\sigma_i \sigma_j} \sigma_i u_i^\mathrm{T} \sigma_j u_j \\ &= \frac{1}{\sigma_i \sigma_j} (\sigma_i u_i)^\mathrm{T} (\sigma_j u_j) \\ &= \frac{1}{\sigma_i \sigma_j} (A v_i)^\mathrm{T} (A v_j) \\ &= \frac{1}{\sigma_i \sigma_j} v_i^\mathrm{T} (A^\mathrm{T} A) v_j \\ &= \frac{1}{\sigma_i \sigma_j} v_i^\mathrm{T} \varepsilon_j v_j \\ &= \frac{1}{\sigma_i \sigma_j} \varepsilon_j v_i^\mathrm{T} v_j \\ &= \frac{1}{\sigma_i \sigma_j} \varepsilon_j 0 \\ &= 0 \text{,} \end{align*} where $\varepsilon_j$ is the eigenvalue of $(A^\mathrm{T} A)$ associated to $v_j$.

  • $\begingroup$ In fact, the key is to understand r eigenvectors(given that $Rank(A)=r$) corresponding to nonzero eigenvalues of $A^TA$ exactly right forms an orthonormal basis of $C(A^T)$ . And notice $A^T(Av/\lambda) =v$ holds for any nonzero eigenvalue $\lambda$. $\endgroup$ – Finley Aug 30 '17 at 6:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.