# Show that $A\varphi_j=\left<A\varphi_j,\varphi_j\right>\varphi_j$ and $A^*A\varphi_j=s_j(A)^2\varphi_j$ for all $j$

Let $A$ be a bounded linear (compact) operator acting on a separable Hilbert space $H$, and let $\varphi_1,\varphi_2,\ldots$ be an orthonormal basis of $H$. I Assume that $|\left< A\varphi_j,\varphi_j\right>|=s_j(A)$ for $j=1,2,\ldots$, where $s_j(A)$ is the j-th singular value, i.e. s-number $s_j(A):=(\lambda_j(A^*A))^{1/2}$. Here, $\lambda_j(A^*A)$ is a non-zero eigenvalue of $A^*A$.

I want to show that $A\varphi_j=\left<A\varphi_j,\varphi_j\right>\varphi_j$ and that $A^*A\varphi_j=s_j(A)^2\varphi_j$ for all $j$.

I know that $A$ is a bounded linear operator on $H$. So I can write for $x\in H$, $x=\sum_j \left<x,\varphi_j\right>\varphi_j$. Hence, $$A\varphi_j=\sum_k\left<A\varphi_j,\varphi_k\right>\varphi_k. \tag{1}$$ So, we can represent (1) as

$$\begin{bmatrix} \left<A\varphi_1,\varphi_1\right> & \left<A\varphi_2,\varphi_1\right> & \dots & \\ \left<A\varphi_1,\varphi_2\right> & \left<A\varphi_2,\varphi_2\right> &\dots & \\ \vdots & \vdots & \ddots &\\ & & & \end{bmatrix} \varphi_j \tag{2}$$

Now, I cannot show that (2) is equal to the first equality and therefore I cannot conclude that the second one holds. Any hints are appreciated.

Update:

I can compute that

$$A^*A=\sum_k s_j(A)^2\left<\cdot,\varphi_k\right>\varphi_k$$

Hence, $$A^*A\varphi_j=s_j(A)^2\left<\varphi_j,\varphi_j\right>\varphi_j=s^2_j(A)\varphi_j.$$

Any hints for the first equality?

• You usually need two orthonormal bases for SVD – Omnomnomnom Jun 27 '16 at 11:16

(You don't say how you got the second equality; since it is not trivial, I'm not sure how you did it and so it is done below)

Since $A^*A$ is positive and compact, it is orthogonally diagonalizable (spectral theorem): $A^*A=U^*D^2U$ for some unitary $U$ and $D$ diagonal with diagonal $s_1(A),s_2(A),\ldots$

Assume $s_1(A)\geq s_2(A)\geq \cdots$

Since $U$ is unitary, we have $$U\varphi_j=\sum_k u_{kj}\,\varphi_k,\ \ \ \ \ \ \ \text{ with } \sum_k|u_{kj}|^2=1$$

You have \begin{align} s_j(A)^2&=|\langle A\varphi_j,\varphi_j\rangle|^2 \leq\langle A\varphi_j,A\varphi_j\rangle\langle\varphi_j,\varphi_j\rangle =\langle A^*A\varphi_j,\varphi_j\rangle\\ \ \\ \tag{1} &=\langle U^*D^2U\varphi_j,\varphi_j\rangle=\langle D^2U\varphi_j,U\varphi_j\rangle\\ \ \\ &=\sum_{k,h}s_k(A)^2\,u_{kj}\,\overline{u_{kj}}\,\langle\varphi_k,\varphi_h\rangle\\ \ \\ &=\sum_k|u_{kj}|^2\,s_k(A)^2. \end{align} When $j=1$, we get from above \begin{align} s_1(A)^2&=|\langle A\varphi_1,\varphi_1\rangle|^2\leq\langle A\varphi_1,A\varphi_1\rangle\,\langle\varphi_1,\varphi_1\rangle=\sum_k|u_{k1}|^2\,s_k(A)^2\\ \ \\ &\leq s_1(A)^2\sum_k|u_{k1}|^2=s_1(A)^2. \end{align} Thus we have equality in Cauchy Schwarz, so $A\varphi_1=\lambda\,\varphi_1$ for some $\lambda$. Doing the inner product with $\varphi_1$ one sees that $\lambda=\langle A\varphi_1,\varphi_1\rangle$. At his stage, if there are repetitions in the list of singular values (i.e., $s_2(A)=s_1(A)$), we can repeat the argument above for those $j$ with $s_j(A)=s_1(A)$ (the key fact is that we are dealing with the largest singular value). So assume that $s_1(A)=s_2(A)=\cdots=s_m(A)$, and $s_{m+1}(A)\ne s_1(A)$.

Now we also know that $\langle A\varphi_1,\varphi_j\rangle=0$ for $j\geq2$ (i.e., the first row of $A$ is zero except at $1,1$). As $s_j(A^*)=s_j(A)$ and $|\langle A\varphi_j,\varphi_j\rangle|=|\langle A^*\varphi_j,\varphi_j\rangle|$, we can repeat the above argument for $A^*$. That is, the first row of $A^*$ consists of zero except at $1,1$, which means that the same happens for the first column of $A$: $$\langle A\varphi_1,\varphi_j\rangle=\langle A\varphi_j,\varphi_1\rangle=0,\ \ \ j\geq2.$$ Then, if we calculate $A^*A$, the $1,1$ entry is $s_1(A)^2$, and the rest of the first row and the first column is zero (and all rows and columns from $1$ to $m$). That is, $$A^*A\varphi_j=s_1(A)^2\varphi_j,\ \ j=1,\ldots,m.$$

Now look at the equality $A^*A=U^*D^2U$, which we can rewrite as $UA^*A=D^2U$. For $j>m$, $k\leq m$, $$s_1(A)^2\langle U\varphi_k,\varphi_j\rangle=\langle UA^*A\varphi_k,\varphi_j\rangle =\langle D^2U\varphi_k,\varphi_j\rangle =\langle U\varphi_k,D^2\varphi_j\rangle =s_j(A)^2\langle U\varphi_k,\varphi_j\rangle.$$ As $s_1(A)\ne s_j(A)$ (because $j>m$), it follows that $\langle U\varphi_k,\varphi_j\rangle=0$. A very similar argument shows that $\langle U\varphi_j,\varphi_k\rangle=0$ for $j>m$, $k\leq m$. We have shown that on $\varphi_1,\ldots,\varphi_m$ the unitary $U$ is diagonal. Then, the restriction of $U$ to $\varphi_{m+1},\varphi_{m+2},\ldots$ is a unitary (in other words, $U$ is the direct sum of two unitaries). The net benefit of all this is that now we can start all over, but with initial index $j=m+1$; now the biggest singular value is $s_2(A)$, and by repeating the whole argument again and again we have that $$A\varphi_j=\langle A\varphi_j,\varphi_j\rangle\,\varphi_j,\ \ \ A^*A\varphi_j=s_1(A)^2\varphi_j$$ for all $j$.

• Great answer! Very helpful, thank you! – Jan Jun 29 '16 at 18:02
• You are welcome. I still have the feeling that it should be possible to get a more straightforward argument, but I don't see how to avoid the Cauchy-Schwarz/convexity argument. – Martin Argerami Jun 29 '16 at 18:49