0
$\begingroup$

Suppose $Rank(A)=r,A{\in}R^{m \times n}$, $v_1,v_2,...,v_r{\in}R^n$ forms an orthonormal basis of $C(A^T)$(i.e. row space of A) and are eigenvectors of $A^TA$.

Now, take $\sigma_1u_1=Av_1,\sigma_2u_2=Av_2,...,\sigma_ru_r=Av_r;\ \sigma_1,\sigma_2,...,\sigma_r$ are positive scaling factors and $u_1,u_2,...,u_r{\in}R^m$ are unit vectors in $C(A)$(column space of A).

Now,we have $A(v_1,v_2,...,v_r)=(u_1,u_2,...,u_r)\begin{bmatrix}\sigma_1& 0 &\cdots &0 \\ 0& \sigma_2 & \cdots &0 \\ \vdots &\vdots &\ddots &\vdots \\0&0&\cdots&\sigma_r\end{bmatrix}$. And denote this formula as $AV=U\Sigma,V{\in}R^{n\times r},U{\in}R^{m\times r},\Sigma$ is a r$\times$r diagonal matrix.

My problem is how to show $u_1,u_2,...,u_r$ is an orthonormal basis of $C(A)$? E.g. $u_1^Tu_2=0$ when take $\sigma_1u_1=Av_1,\sigma_2u_2=Av_2$.

Above thread comes from MIT's opencourse 18.06 linear algebra lecture 29

$\endgroup$
10
  • $\begingroup$ The vectors $v_i$ must be chosen in a special way. $\endgroup$
    – littleO
    Commented Aug 30, 2017 at 2:15
  • $\begingroup$ @littleO : Can you state the trick concretely? $\endgroup$
    – Finley
    Commented Aug 30, 2017 at 2:32
  • $\begingroup$ @Finley This is a very strange way to present the idea of finding an SVD. Where is this question coming from; that is, what have you learned/read most recently? Have you tried reading a textbook and finding the SVD with the method outline there? $\endgroup$
    – Ben Grossmann
    Commented Aug 30, 2017 at 2:39
  • $\begingroup$ @Finley how about this: are you at least aware that the $\sigma_i$ must be the square roots of the eigenvalues of $A^TA$? $\endgroup$
    – Ben Grossmann
    Commented Aug 30, 2017 at 2:42
  • $\begingroup$ Here's a relevant thread that explains one way of understanding the SVD: math.stackexchange.com/questions/1737637/… $\endgroup$
    – littleO
    Commented Aug 30, 2017 at 2:42

1 Answer 1

Reset to default
1
$\begingroup$

Assuming everything in the problem statement, ... \begin{align*} u_i^\mathrm{T} u_j &= \frac{1}{\sigma_i \sigma_j} \sigma_i \sigma_j u_i^\mathrm{T} u_j \\ &= \frac{1}{\sigma_i \sigma_j} \sigma_i u_i^\mathrm{T} \sigma_j u_j \\ &= \frac{1}{\sigma_i \sigma_j} (\sigma_i u_i)^\mathrm{T} (\sigma_j u_j) \\ &= \frac{1}{\sigma_i \sigma_j} (A v_i)^\mathrm{T} (A v_j) \\ &= \frac{1}{\sigma_i \sigma_j} v_i^\mathrm{T} (A^\mathrm{T} A) v_j \\ &= \frac{1}{\sigma_i \sigma_j} v_i^\mathrm{T} \varepsilon_j v_j \\ &= \frac{1}{\sigma_i \sigma_j} \varepsilon_j v_i^\mathrm{T} v_j \\ &= \frac{1}{\sigma_i \sigma_j} \varepsilon_j 0 \\ &= 0 \text{,} \end{align*} where $\varepsilon_j$ is the eigenvalue of $(A^\mathrm{T} A)$ associated to $v_j$.

$\endgroup$
1
  • $\begingroup$ In fact, the key is to understand r eigenvectors(given that $Rank(A)=r$) corresponding to nonzero eigenvalues of $A^TA$ exactly right forms an orthonormal basis of $C(A^T)$ . And notice $A^T(Av/\lambda) =v$ holds for any nonzero eigenvalue $\lambda$. $\endgroup$
    – Finley
    Commented Aug 30, 2017 at 6:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .