# SVD(Singular Value Decomposition)

I have several questions regarding SVD. I know that we have to find eigen vectors of A^T A and go from there but what I don't understand is how do you know that the eigen vectors of A^T A will also be eigen vectors of A? Also, the book says if there are n eigen vectors for A^T A then, for some r in 1<= r <= n, Av1... Avr will form a orthogonal basis for columns of A. Again, how do we know for certain that A^TA will have more eigen vectors than A? A might have rank of 15 and A^TA might only have 4 eigen vectors what then?

• You're interested in the singular vectors of $A$ rather than the eigenvectors of $A$. Since $A$ is potentially not square, it might not even have eigenvectors. – Brian Borchers Dec 9 '16 at 17:11

what I don't understand is how do you know that the eigen vectors of A^T A will also be eigen vectors of A?

The eigenvalues of $A^TA$ will generally not be the eigenvalues of $A$; they will be the singular values of $A$. Similarly, the eigenvectors of $A^TA$ are singular vectors of $A$, which need not coincide with the eigenvectors of $A$. Notably: if $A$ isn't square, then $A$ doesn't have eigenvalues/eigenvectors, but it will still have singular values/singular vectors.

how do we know for certain that A^TA will have more eigen vectors than A?

$A^TA$ is a symmetric matrix. By the spectral theorem, it is orthogonally diagonalizable, which is to say that it necessarily has an orthonormal basis of eigenvectors. This is true regardless of the properties of $A$.

A might have rank of 15 and A^TA might only have 4 eigen vectors what then?

The rank of a matrix generally doesn't tell you whether that matrix is diagonalizable (i.e. has enough eigenvectors).

• How does "A^TA is a symmetric matrix. By the spectral theorem, it is necessarily diagonalizable, which is to say that it necessarily has an orthonormal basis of eigenvectors. This is true regardless of the properties of AA. " this translate to A^T A having more eigen vectors than A? – Hello Dec 9 '16 at 17:16
• If you can be very clear about what you mean by "$A^TA$ has more eigenvectors than $A$", then I can answer that question. Maybe you can add an example to your question of matrices $A$ and $B$ such that $A$ "has more eigenvectors than" $B$. – Omnomnomnom Dec 9 '16 at 17:18
• I can't think of an example on the spot but.. just imagine if A had 2 eigen vectors.. How could we be sure that A^TA would have enough vectors in its orthonormal basis that some of its eigen vectors could form an orthogonal basis for Col A? What if it runs out – Hello Dec 9 '16 at 17:27
• You mean to say, then, that $A$ has two linearly independent eigenvectors. Again, saying matrix is diagonalizable is precisely the same as saying that it has enough eigenvectors to build a basis. The spectral theorem says that because $A^TA$ is symmetric, it has enough eigenvectors to build a basis (of $\Bbb R^m$) and we can make that basis orthonormal. It so happens that we can use some of those eigenvectors to build a basis of $col(A) \subset \Bbb R^m$. – Omnomnomnom Dec 9 '16 at 17:30
• umm. isn't eigenvectors a basis of R^n not R^m? A^TA is n x n matrix.. – Hello Dec 9 '16 at 17:37

The book doesn't say "more eigenvectors than $A$".

If $A$ is $m \times n$, then $A^T A$ is $n \times n$. Being a real symmetric matrix, $A^T A$ has $n$ linearly independent eigenvectors. The rank of $A$ is at most $n$ (which is the number of columns), and also at most $m$ (the number of rows). In particular, it's impossible for $A$ to have rank $15$ if $A^T A$ has only $4$ linearly independent eigenvectors.