Let $A$ be a bounded linear (compact) operator acting on a separable Hilbert space $H$, and let $\varphi_1,\varphi_2,\ldots$ be an orthonormal basis of $H$. I Assume that $|\left< A\varphi_j,\varphi_j\right>|=s_j(A)$ for $j=1,2,\ldots$, where $s_j(A)$ is the j-th singular value, i.e. s-number $s_j(A):=(\lambda_j(A^*A))^{1/2}$. Here, $\lambda_j(A^*A)$ is a non-zero eigenvalue of $A^*A$.

I want to show that $A\varphi_j=\left<A\varphi_j,\varphi_j\right>\varphi_j$ and that $A^*A\varphi_j=s_j(A)^2\varphi_j$ for all $j$.

I know that $A$ is a bounded linear operator on $H$. So I can write for $x\in H$, $x=\sum_j \left<x,\varphi_j\right>\varphi_j$. Hence, $$ A\varphi_j=\sum_k\left<A\varphi_j,\varphi_k\right>\varphi_k. \tag{1} $$ So, we can represent (1) as

$$ \begin{bmatrix} \left<A\varphi_1,\varphi_1\right> & \left<A\varphi_2,\varphi_1\right> & \dots & \\ \left<A\varphi_1,\varphi_2\right> & \left<A\varphi_2,\varphi_2\right> &\dots & \\ \vdots & \vdots & \ddots &\\ & & & \end{bmatrix} \varphi_j \tag{2} $$

Now, I cannot show that (2) is equal to the first equality and therefore I cannot conclude that the second one holds. Any hints are appreciated.


I can compute that

$$ A^*A=\sum_k s_j(A)^2\left<\cdot,\varphi_k\right>\varphi_k $$

Hence, $$ A^*A\varphi_j=s_j(A)^2\left<\varphi_j,\varphi_j\right>\varphi_j=s^2_j(A)\varphi_j. $$

Any hints for the first equality?

  • $\begingroup$ You usually need two orthonormal bases for SVD $\endgroup$ – Omnomnomnom Jun 27 '16 at 11:16

(You don't say how you got the second equality; since it is not trivial, I'm not sure how you did it and so it is done below)

Since $A^*A$ is positive and compact, it is orthogonally diagonalizable (spectral theorem): $A^*A=U^*D^2U$ for some unitary $U$ and $D$ diagonal with diagonal $s_1(A),s_2(A),\ldots$

Assume $s_1(A)\geq s_2(A)\geq \cdots$

Since $U$ is unitary, we have $$ U\varphi_j=\sum_k u_{kj}\,\varphi_k,\ \ \ \ \ \ \ \text{ with } \sum_k|u_{kj}|^2=1$$

You have \begin{align} s_j(A)^2&=|\langle A\varphi_j,\varphi_j\rangle|^2 \leq\langle A\varphi_j,A\varphi_j\rangle\langle\varphi_j,\varphi_j\rangle =\langle A^*A\varphi_j,\varphi_j\rangle\\ \ \\ \tag{1} &=\langle U^*D^2U\varphi_j,\varphi_j\rangle=\langle D^2U\varphi_j,U\varphi_j\rangle\\ \ \\ &=\sum_{k,h}s_k(A)^2\,u_{kj}\,\overline{u_{kj}}\,\langle\varphi_k,\varphi_h\rangle\\ \ \\ &=\sum_k|u_{kj}|^2\,s_k(A)^2. \end{align} When $j=1$, we get from above \begin{align} s_1(A)^2&=|\langle A\varphi_1,\varphi_1\rangle|^2\leq\langle A\varphi_1,A\varphi_1\rangle\,\langle\varphi_1,\varphi_1\rangle=\sum_k|u_{k1}|^2\,s_k(A)^2\\ \ \\ &\leq s_1(A)^2\sum_k|u_{k1}|^2=s_1(A)^2. \end{align} Thus we have equality in Cauchy Schwarz, so $A\varphi_1=\lambda\,\varphi_1$ for some $\lambda$. Doing the inner product with $\varphi_1$ one sees that $\lambda=\langle A\varphi_1,\varphi_1\rangle$. At his stage, if there are repetitions in the list of singular values (i.e., $s_2(A)=s_1(A)$), we can repeat the argument above for those $j$ with $s_j(A)=s_1(A)$ (the key fact is that we are dealing with the largest singular value). So assume that $s_1(A)=s_2(A)=\cdots=s_m(A)$, and $s_{m+1}(A)\ne s_1(A)$.

Now we also know that $\langle A\varphi_1,\varphi_j\rangle=0$ for $j\geq2$ (i.e., the first row of $A$ is zero except at $1,1$). As $s_j(A^*)=s_j(A)$ and $|\langle A\varphi_j,\varphi_j\rangle|=|\langle A^*\varphi_j,\varphi_j\rangle|$, we can repeat the above argument for $A^*$. That is, the first row of $A^*$ consists of zero except at $1,1$, which means that the same happens for the first column of $A$: $$ \langle A\varphi_1,\varphi_j\rangle=\langle A\varphi_j,\varphi_1\rangle=0,\ \ \ j\geq2. $$ Then, if we calculate $A^*A$, the $1,1$ entry is $s_1(A)^2$, and the rest of the first row and the first column is zero (and all rows and columns from $1$ to $m$). That is, $$ A^*A\varphi_j=s_1(A)^2\varphi_j,\ \ j=1,\ldots,m. $$

Now look at the equality $A^*A=U^*D^2U$, which we can rewrite as $UA^*A=D^2U$. For $j>m$, $k\leq m$, $$ s_1(A)^2\langle U\varphi_k,\varphi_j\rangle=\langle UA^*A\varphi_k,\varphi_j\rangle =\langle D^2U\varphi_k,\varphi_j\rangle =\langle U\varphi_k,D^2\varphi_j\rangle =s_j(A)^2\langle U\varphi_k,\varphi_j\rangle. $$ As $s_1(A)\ne s_j(A)$ (because $j>m$), it follows that $\langle U\varphi_k,\varphi_j\rangle=0$. A very similar argument shows that $\langle U\varphi_j,\varphi_k\rangle=0$ for $j>m$, $k\leq m$. We have shown that on $\varphi_1,\ldots,\varphi_m$ the unitary $U$ is diagonal. Then, the restriction of $U$ to $\varphi_{m+1},\varphi_{m+2},\ldots$ is a unitary (in other words, $U$ is the direct sum of two unitaries). The net benefit of all this is that now we can start all over, but with initial index $j=m+1$; now the biggest singular value is $s_2(A)$, and by repeating the whole argument again and again we have that $$A\varphi_j=\langle A\varphi_j,\varphi_j\rangle\,\varphi_j,\ \ \ A^*A\varphi_j=s_1(A)^2\varphi_j$$ for all $j$.

  • $\begingroup$ Great answer! Very helpful, thank you! $\endgroup$ – Jan Jun 29 '16 at 18:02
  • $\begingroup$ You are welcome. I still have the feeling that it should be possible to get a more straightforward argument, but I don't see how to avoid the Cauchy-Schwarz/convexity argument. $\endgroup$ – Martin Argerami Jun 29 '16 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.