3
$\begingroup$

If $L$ is a linear operator acting on a hilbert space $H$ of dimension $n$ ( $L: H \to H$ ), then I know the following

  1. If $L$ is normal operator then in some orthonormal basis $B$, matrix representation of $L$, $[L]_{B}$ will be a diagonal matrix.
  2. If sum of dimensions of eigen spaces of $L$ is equal to $n$ then in some non-orthonormal basis matrix representation of $L$ is diagonal.
  3. In general for any $L$, if its matrix representation in a basis $B$ is $[L]_B$ then $[L_B] = UDV$ where $U,V$ are unitary matrices and $D$ a diagonal matrix.

I can see that point $1$ and $2$ are the same thing ie if $L$ is normal operator. Is there any other relationship between SVD decomposition and diagonalizability ?

$\endgroup$
6
  • $\begingroup$ Could you clarify the notation $[L]_B$? $\endgroup$ May 7, 2015 at 18:33
  • 2
    $\begingroup$ I don't think so. Every matrix has an SVD, not all are diagonalisable. Note that diagonalisable means that you can find a basis change for both the range and domain that results in a diagonal operator, whereas the SVD has different bases for the domain and range. $\endgroup$
    – copper.hat
    May 7, 2015 at 18:33
  • $\begingroup$ $[L]_{B}$ stands for matrix representation of linear operator $L$ in basis $B$. $\endgroup$ May 7, 2015 at 18:43
  • 2
    $\begingroup$ For a matrix $A$, its eigenvalue decomposition and SVD are the same if and only if $A$ is a symmetric positive definite matrix. I'm too lazy to translate that into the $[L]_B$ notation. $\endgroup$ May 7, 2015 at 18:59
  • 1
    $\begingroup$ the more common terms are "positive semi-definite" or, in the context of operator theory, just "positive" (as opposed to strictly positive). $\endgroup$ May 7, 2015 at 19:28

2 Answers 2

4
$\begingroup$

I think the best way to describe the relationship between SVD of a matrix (I'll just use $A$) and diagonalizability, and what makes it possible for every matrix to have a SVD, is because it is more closely related to the eigendecomposition of $AA^*$ and $A^*A$ (these matrices are positive semidefinite and is therefore always unitarily diagonalizable) than to $A$ itself. Notice that for $A=UDV$ we have \begin{equation} AA^*=UDVV^*D^*U^*=UD^2U^*\end{equation} and similarly we have $A^*A=V^*D^2V$. So $D$ is actually the square root of the eigenvalues of $AA^*$ and $A^*A$. Furthermore $U$ consists of eigenvectors for $AA^*$ and similarly $V$ consists of eigenvectors for $A^*A$.

$\endgroup$
2
  • $\begingroup$ thanks did not think in terms of $AA^*$, is an interesting way to look. $\endgroup$ May 7, 2015 at 20:07
  • 1
    $\begingroup$ pleasure. it becomes even more relevant if the matrix $A$ is not square, ie your transformation is between spaces with different dimensions. $\endgroup$ May 9, 2015 at 6:03
3
$\begingroup$

For any normal matrix, we can obtain an SVD from any spectral decomposition (i.e. orthonormal diagonalization).

Let $L$ be normal. There exists a unitary matrix (orthonormal change of basis) $V$ such that $L = VDV^*$, where $$ D = \pmatrix{\lambda_1 \\ & \ddots \\ & & \lambda_n} $$ With $\lambda_k \in \Bbb C$. We define the function $$ \operatorname{sgn}(x) = \begin{cases} \frac{x}{|x|} & x \neq 0\\ 1 & x = 0 \end{cases} $$ We then write $D = U'\Sigma$, where $$ U' = \pmatrix{ \operatorname{sgn}(\lambda_1)\\ &\ddots\\ && \operatorname{sgn}(\lambda_n) }, \quad \Sigma = \pmatrix{ |\lambda_1|\\ & \ddots \\ &&|\lambda_n| } $$ Thus, we have $L = VU'\Sigma V^*$. Define $U = VU'$. Note that $U$ is the product of unitary matrix, and is therefore itself unitary. It follows that $L = U\Sigma V^*$ is a SVD of $L$.


If the matrix is not normal, then there is no such direct connection between diagonalization and SVD. That is, only the things that we can do with orthonormal changes of basis allow us to deduce the SVD.

We do have some interesting general relationships between eigenvalues and singular values, though. Suppose $s_1,\dots,s_n$ are the singular values in decreasing order and $\lambda_1,\dots,\lambda_n$ are the eigenvalues in order of decreasing magnitude. Then Weyl's majorant theorem (Bhatia "Matrix Analysis", p. 42) states, among other things, that for any $p \geq 0$ and $k = 1,\dots,n$, we have $$ \prod_{j=1}^k |\lambda_j| \leq \prod_{j=1}^k s_j \\ \sum_{j=1}^k |\lambda_j|^p \leq \sum_{j=1}^k s_1^p $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .