# SVD Question on MIT OCW Linear Algebra Lecture 29

Apparently Gilbert Strang Linear Algebra lecture 29 does SVD on $A$ being calculated using

$$A^TA = V \Sigma^T \Sigma V^{T}$$

and

$$AA^T = U\Sigma \Sigma^TU^T$$

Example is

$$A = \left[\begin{array}{rr} 4 & 4 \\ -3 & 3 \end{array}\right]$$

$$A^TA = \left[\begin{array}{rr} 4 & -3 \\ 4 & 3 \end{array}\right] \left[\begin{array}{rr} 4 & 4 \\ -3 & 3 \end{array}\right] = \left[\begin{array}{cc} 25 & 7 \\ 7 & 25 \end{array}\right]$$

The eigenvectors are

$$\left[\begin{array}{r} 1 \\ 1 \end{array}\right], \left[\begin{array}{r} 1 \\ -1 \end{array}\right]$$

Eigenvalues are 32,18.

Normalized

$$\left[\begin{array}{r} 1 / \sqrt{ 2} \\ 1/ \sqrt{ 2} \end{array}\right], \left[\begin{array}{r} 1/ \sqrt{ 2} \\ -1/ \sqrt{ 2} \end{array}\right]$$

Same for $V$

$$A^TA = \left[\begin{array}{rr} 4 & 4 \\ -3 & 3 \end{array}\right] \left[\begin{array}{rr} 4 & -3 \\ 4 & 3 \end{array}\right] = \left[\begin{array}{rr} 32 & 0 \\ 0 & 18 \end{array}\right]$$

Eigenvectors

$$\left[\begin{array}{r} 1 \\ 0 \end{array}\right], \left[\begin{array}{r} 0 \\ 1 \end{array}\right]$$

Eigenvalues are the same 32,18.

So the whole thing is

$$\underbrace{ \left[\begin{array}{rr} 4 & 4 \\ -3 & 3 \end{array}\right] }_{A} = \underbrace{ \left[\begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right] }_{U} \underbrace{ \left[\begin{array}{rr} \sqrt{ 32} & 0 \\ 0 & \sqrt{ 18} \end{array}\right] }_{\Sigma} \underbrace{ \left[\begin{array}{rr} 1/\sqrt{ 2} & 1/\sqrt{ 2} \\ 1/\sqrt{ 2} & -1/\sqrt{ 2} \end{array}\right] }_{V^T}$$

But the calculation on the RHS does not result in $A$ seen in LHS. I followed the logic, and reach the same results in the steps, however the final result is wrong. I thought the order of evectors could be wrong, played with those, but the answer is still wrong. Quick Python code to verify

import scipy.linalg as lin
import numpy as np

print '\nA.T*A\n'
a = np.array([[4,4],[-3,3]])
print np.dot(a.T,a)

print '\nEig of A.T*A\n'
a = np.array([[25,7],[7,25]])
w,vl =  lin.eig(a)
print w
print vl

print '\nEig of A*A.T\n'
a = np.array([[32,0],[0,18]])
w,vl =  lin.eig(a)
print w
print vl

print '\nVerify\n'
a1 = np.array([[1,0],[0,1]])
a2 = np.array([[np.sqrt(32),0],[0,np.sqrt(18)]])
a3 = np.array([[1./np.sqrt(2),1./np.sqrt(2)],
[1./np.sqrt(2),-1./np.sqrt(2)]])

print np.dot(a1,np.dot(a2,a3))


The result comes out as

[[ 4.  4.]
[ 3. -3.]]


Almost there, but not exactly.

Any ideas?

-

These worked

$$\left[\begin{array}{r} 1 \\ 1 \end{array}\right], \left[\begin{array}{r} -1 \\ 1 \end{array}\right]$$

Strang lecture notes.

-
(-1) for saying the eigenvectors are 'wrong'... – TMM Feb 29 '12 at 14:12
I am sorry, why is that bad? – BB_ML Feb 29 '12 at 14:48
Why are his eigenvectors 'wrong'? Are you saying they are not eigenvectors of $A^TA$? – TMM Feb 29 '12 at 15:04
If I could I would downvote that comment too. The point of my previous comment was to make you reconsider your remark, but apparently I have to be more direct. Please review your basics on vector spaces and figure out why your statement, i.e. "Strang's eigenvectors were wrong", is false. (And I should not have to mention this, but $A^T A \cdot [1, -1] = 18 \cdot [1, -1]$, proving that $[1, -1]$ is an eigenvector of $A^T A$ with eigenvalue $18$.) – TMM Feb 29 '12 at 15:54
Like I said, the eigenvector you claim to be wrong can easily be verified to be an eigenvector of $A^TA$. Does that not convince you that his 'wrong' eigenvector is in fact correct? And people learn more if they discover things for themselves instead of others just giving them the answers. Hint: The set of eigenvectors (for a certain eigenvalue) form what's called an eigenspace, which (with the exception of the zero vector) forms a vector space. – TMM Feb 29 '12 at 16:16