0
$\begingroup$

I was watching Singular Value Decomposition Lecture by Gilbert Strang. He takes two orthonormal vectors $v_1$, $v_2$. Let $\sigma_1$$u_1$=A$v_1$ and $\sigma_2$$u_2$=A$v_2$. He takes $u_1$ is orthogonal to $u_2$. How? How to prove this?

Edit1: Adding link to video lecture https://www.youtube.com/watch?v=Nx0lRBaXoz4 (Start from 3:30 mins)

$\endgroup$
  • $\begingroup$ You're assuming $\sigma_1\ne \sigma_2$ here. Consider $\langle Mv_1,u_2\rangle$, and use $Mv_1=\sigma_1 u_1$ and $M^*u_2 = \sigma_2 v_2$. $\endgroup$ – Ted Shifrin May 12 '18 at 5:01
  • $\begingroup$ @TedShifrin, I didn't get it. Can you elaborate? Also 1 more thing. Does $M^T$$u_2$=$v_2$? $\endgroup$ – Nagabhushan S N May 12 '18 at 8:44
0
$\begingroup$

Recall that the SVD is given by $ A = U \Sigma V ^ T $, hence $ A ^ T A = ( U \Sigma V ^ T) (U \Sigma V ^ T)^T = U \Sigma ^ 2U ^T$. Namely, the columns of $U$ are the eigenvectors of $A ^ T A$, and as $A^TA$ is symmetric it is can be diagonilzed by an orthogonal matrix ($U$).

$\endgroup$
  • $\begingroup$ I ran into the problem while deriving SVD! $\endgroup$ – Nagabhushan S N May 13 '18 at 22:08
  • $\begingroup$ $u_1$ and $u_2$ are the first and the second column of $U$, which is orthogonal, hence by definition $u_1 ^ T u_2 = 0$. And the proof (that $U$ is orthogonal follows from my answer). $\endgroup$ – V. Vancak May 13 '18 at 22:12
  • $\begingroup$ How do you say $U$ is orthogonal? In your answer, you've used SVD. But as I said before, Gilbert Strang uses the fact that $U$ is orthogonal to prove SVD. So, there should be another way to prove that $U$ is orthogonal without using SVD. I'm stuck there. $\endgroup$ – Nagabhushan S N May 14 '18 at 14:29
0
$\begingroup$

I found an explanation here and here. It explains that we choose $v_1$ such that $Av_1$ is maximum. Then we choose $v_2$ such that $v_2 \perp v_1$ and $Av_2$ is maximum. In that case, the above referred answers prove that $$Av_1 \perp Av_2$$ We choose $$u_1=Av_1/||Av_1||$$ $$u_2=Av_2/||Av_2||$$ Hence $$ u_1 \perp u_2$$

I guess Gilbert Strang omitted/missed this explanation in that lecture.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.