# Proof of the CS (cosine-sine) matrix decomposition

The CS decomposition is a way to write the singular value decomposition of a matrix with orthonormal columns. More specifically, taking the notation from these notes (pdf alert), consider a $$(n_1+n_2)\times p$$ matrix $$Q$$, with $$Q=\begin{bmatrix}Q_1 \\ Q_2\end{bmatrix},$$ where $$Q_1$$ has dimensions $$n_1\times p$$ and $$Q_2$$ has dimensions $$n_2\times p$$. Assume $$Q$$ has orthonormal columns, that is, $$Q_1^\dagger Q_1+Q_2^\dagger Q_2=I$$.

Then the CS decomposition essentially tells us that the SVDs of $$Q_1$$ and $$Q_2$$ are related. More specifically, there are unitaries $$V, U_1, U_2$$ such that \begin{aligned} U_1^\dagger Q_1 V=\operatorname{diag}(c_1,...,c_p), \\ U_2^\dagger Q_2 V=\operatorname{diag}(s_1,...,s_q), \end{aligned} with $$c_i^2+s_i^2=1$$ (from which the name of the decomposition comes). As far as I understand, this means that there is a set of orthonormal vectors $$\{v_k\}_k$$ such that both $$\{Q_1 v_k\}_k$$ and $$\{Q_2 v_k\}$$ are orthogonal sets of vectors (with some relations between their norms).

To prove that this is the case, I start by writing down the SVDs of $$Q_1$$ and $$Q_2$$, which tell us that there are unitaries $$U_1, U_2, V_1, V_2$$, and diagonal positive matrices $$D_1, D_2$$, such that \begin{aligned} Q_1= U_1 D_1 V_1^\dagger, \\ Q_2= U_2 D_2 V_2^\dagger. \end{aligned} The condition $$Q_1^\dagger Q_1+Q_2^\dagger Q_2=I$$ then translates into $$V_1 D_1^2 V_1^\dagger + V_2 D_2^2 V_2^\dagger=I.$$ Denoting with $$v^{(i)}_k$$ the $$k$$-th column of $$V_i$$, and $$P^{(i)}_k\equiv v^{(i)}_k v^{(i)*}_k$$ the associated projector, this condition can be seen to be equivalent to $$\sum_k (d^{(1)}_k)^2 P_k^{(1)}+\sum_k (d^{(2)}_k)^2 P_k^{(2)}=I,\tag A$$ where $$d^{(i)}_k\equiv (D_i)_{kk}$$.

Now, however, I'm a bit stuck into how to prove (A). It seems a generalisation of the things proved in this post and links therein, which show that if a sum of projectors gives the identity then the projectors must be orthogonal, but I'm not sure how to prove this in this case.

To get to $$(A)$$ and proceed from there to show this equation corresponds to $$c_i^2 + s_i^2 = 1$$, we need to get to $$V_1^\dagger = V_2^\dagger$$.

To get there consider the "$$QR$$" decomposition of $$Q_2V_1$$ matrix. We can write it as: $$Q_2V_1 = U_2R\\ Q_2 = U_2RV_1^\dagger$$ where $$U_2$$ is an orthogonal matrix and $$R$$ is an upper diagonal matrix.

We have $$Q_2Q_2^\dagger = I$$ ($$Q_2$$ is full column rank with orthonormal columns). Therefore: $$(U_2RV_1^\dagger)(VR^\dagger U_2^\dagger) = I \\ U_2 R R^\dagger U_2^\dagger = I \\ R R^\dagger = U_2^\dagger U_2 = I \\$$

Hence $$R$$ must be a diagonal matrix, lets call it $$D_2$$. Rewriting $$Q_2$$ we get $$Q_2V_1 = U_2D_2 \\ Q_2 = U_2D_2V_1^\dagger \\$$ which is same the SVD of $$Q_2 = U_2D_2V_2^\dagger$$. Therefore $$V_2^\dagger = V_1^\dagger$$.

Now using the condition $$Q_1^\dagger Q_1+Q_2^\dagger Q_2=I$$, we get: $$(V_1D_1^\dagger U_1^\dagger)(U_1D_1V_1^\dagger) + (V_1D_2^\dagger U_2^\dagger)(U_2D_2V_1^\dagger)) = I \\ V_1 D_1^\dagger D_1 V_1^\dagger + V_1 D_2^\dagger D_2 V_1^\dagger = I \\ V_1(D_1^\dagger D_1 + D_2^\dagger D_2)V_1^\dagger = I \\ D_1^\dagger D_1 + D_2^\dagger D_2 = V_1^\dagger V_1 = I \\ \sum_k (d^{(1)}_k)^2 +\sum_k (d^{(2)}_k)^2 = I \\$$

if $$d^{(1)}_i = c_i$$ and $$d^{(2)}_i = s_i$$, then $$c_i^2 + s_i^2 = 1$$ for $$i = 1, 2, .., p$$

• how do you know that $Q_2$ has orthonormal rows (i.e. $Q_2 Q_2^\dagger=I$)? You could have e.g. $Q_1=I_2$ and $Q_2=0$ the zero $2\times 2$ matrix, and then this would not be true – glS Dec 1 '19 at 14:06