12
$\begingroup$

Let $USV^T$ be a singular value decomposition of matrix $A$. In the textbook "Linear Algebra and Its Applications" by D. C. Lay et. al., where SVD is introduced, it says that "the columns of $U$ in such a decomposition are called left singular vectors of $A$, and the columns of $V$ are called right singular vectors of $A$." But it does not make any connections with the eigenvectors of $A^T\!A$. It also says that "the matrices $U$ and $V$ are not uniquely determined by $A$.

But in this web page it says that "the eigenvectors of $A^T\!A$ make up the columns of $V$, the eigenvectors of $AA^T$ make up the columns of $U$."

I'm confused about the relationship between the left and right singular vectors (that is columns of $U$ and $V$) and the eigenvectors of $A^T\!A$ and $AA^T$. Any clarification is appreciated.

$\endgroup$
1
  • 1
    $\begingroup$ One observation: Suppose that $A$ is a real $n \times n$ matrix and $A = U \Sigma V^T$, where $U$ and $V$ are orthogonal matrices and $\Sigma$ is diagonal. (I'm assuming $A$ is square just for simplicity.) Then $A^T A = V \Sigma^T U^T U \Sigma V^T = V \Sigma^2 V^T$, which implies that $A^T A V = V \Sigma^2$. This equation, when read "column by column", tells us that the columns of $V$ are eigenvectors of $A^T A$. $\endgroup$
    – littleO
    Commented Jan 12, 2021 at 10:29

1 Answer 1

9
$\begingroup$

Let $A=UDV^*$. Then $$A^*A=VDU^*UDV^*=VD^2V^*\implies A^*AV=VD^2$$ $$AA^*=UDV^*VDU^*=UD^2U^*\implies AA^*U=UD^2$$ When a matrix is right-multiplied by a diagonal matrix, each column is multiplied by the diagonal term: $$[\mathbf{b}_1,\ldots,\mathbf{b}_n]\begin{pmatrix}\sigma^2_1&\\&\ddots\\&&\sigma_n^2\end{pmatrix}=[\sigma^2_1\mathbf{b}_1,\ldots,\sigma^2_n\mathbf{b}_n]$$

Hence the right-hand expressions are exactly the eigenvalue equations, $$A^*A\mathbf{v}_i=\sigma_i^2\mathbf{v}_i,\qquad {AA}^*\mathbf{u}_i=\sigma_i^2\mathbf{u}_i$$

$\endgroup$
2
  • $\begingroup$ I see, thanks for the clarification, one more related question: why "the matrices 𝑈 and 𝑉 are not uniquely determined by 𝐴."? $\endgroup$
    – Sanyo Mn
    Commented Jan 12, 2021 at 11:43
  • 1
    $\begingroup$ @SanyoMn $U$ and $V$ need not be unique for three reasons: (i) the singular values and their corresponding singular vectors can be permuted. (ii) each singular vector is unique only up to a sign, (iii) repeated singular values, especially the zero ones, do not have unique singular vectors - they can be rotated around. $\endgroup$ Commented Jan 12, 2021 at 12:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .