1
$\begingroup$

I am a bit confused with my calculation results of SVD for matrix A.

$$ A= \left( \begin{array}{ccc} 4 & 2 & 1\\ 5 & 3 & -10\\ 9 & -12 & 11 \end{array} \right) \\\\ A = UDV^{T} \mbox{ for real matrix} $$

As I know that $U$ contains orthonormal eigenvectors of $AA^{T}$ and $V$ contains orthonormal eigenvectors of $A^{T}A$. Using python I have found eigenvalues and actual SVD:

import numpy as np

A = np.matrix([[4, 2, 1],
               [5, 3, -10],
               [9, -12, 11]])

# Calculating SVD via eigenvectors
aaeigval, aaeigvec = np.linalg.eig(A*A.T)
ateigval, ateigvec = np.linalg.eig(A.T*A)

MU = aaeigvec
MD = np.diag(np.sqrt(aaeigval))
MV = ateigvec.T

# Calculating SVD via embedded function
U, d, V = np.linalg.svd(A)
D = np.diag(d)

print("SVD MATRICES VIA NP.LINALG.SVD")
print(U)
print(D)
print(V)

print("\r\nSVD MATRICES VIA EIGENVECTORS")
print(MU)
print(MD)
print(MV)

The output is:

SVD MATRICES VIA NP.LINALG.SVD
[[-0.04227372 -0.27992453 -0.95909081]
 [ 0.36803649 -0.89680917  0.24552484]
 [-0.92884987 -0.34260117  0.1409339 ]]

[[ 19.67398841   0.           0.        ]
 [  0.          10.02047314   0.        ]
 [  0.           0.           3.67753968]]

[[-0.33996976  0.61836777 -0.70854912]
 [-0.86694055  0.08591785  0.4909503 ]
 [-0.36446486 -0.78117822 -0.50687863]]

SVD MATRICES VIA EIGENVECTORS
[[-0.95909081  0.27992453  0.04227372]
 [ 0.24552484  0.89680917 -0.36803649]
 [ 0.1409339   0.34260117  0.92884987]]

[[  3.67753968   0.           0.        ]
 [  0.          10.02047314   0.        ]
 [  0.           0.          19.67398841]]

[[-0.33996976  0.61836777 -0.70854912]
 [ 0.86694055 -0.08591785 -0.4909503 ]
 [ 0.36446486  0.78117822  0.50687863]]

As you can see some columns are flipped but that is not the problem. The problem is in different signs in columns. For instance column 1 of $U$ and column 3 of $MU$ (they have to be the same eigenvector) have different signs at their elements. Of course true decomposition is that of svd function. However I am still missing where I am wrong.

$\endgroup$

1 Answer 1

3
$\begingroup$

Hint: If $\mathbf{v}$ is a unit-norm eigen vector of a matrix $\mathbf{A}$ corresponding to the eigen value of $\lambda$ then $-\mathbf{v}$ is also a unit-norm eigenvector of $\mathbf{A}$. In particular, $$\mathbf{A}\mathbf{v}=\lambda \mathbf{v} \implies \mathbf{A}(-\mathbf{v})=\lambda(-\mathbf{v}).$$

$\endgroup$
5
  • $\begingroup$ Oh, sure. I forgot that V and MV matrices are transposed therefore their rows(and not columns) must lie in the same span that is what confused me. However I am still missing why the result of multiplication of my matrices does not match A. $\endgroup$
    – Long Smith
    Aug 14, 2017 at 6:42
  • $\begingroup$ What happens if you instead of finding a solution of $x+1=0$ find a solution of $x^2-1=0$. Obviously, $x=\pm 1$ satisfy $x^2-1=0$. But, if you use only one solution $x=1$ instead of $x=-1$ then you might get an incorrect answer. $\endgroup$
    – Math Lover
    Aug 14, 2017 at 17:11
  • $\begingroup$ If you had computed $V^T$ as $D^{-1} U^T A$ then no matter what signs the eigen vectors had in $U$, $A=U D V^T$. $\endgroup$
    – Math Lover
    Aug 14, 2017 at 17:17
  • $\begingroup$ Thanks! At first your last tip looked like a cheating for me. However having thought about it, I realized that it really has nothing wrong with it. $\endgroup$
    – Long Smith
    Aug 15, 2017 at 15:41
  • $\begingroup$ @LongSmith Just to clarify one point to you. When $A$ is fat/tall, the matrix $D$ is not square. In that case, $A U^T = D V^T$ or $A V = U D$ can be used to compute $V^T$ or $U$, respectively. $\endgroup$
    – Math Lover
    Aug 15, 2017 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.