Matrix projection along another matrix I vaguely have this idea mentioned to me by somebody, but neither can I find anything exactly like it anywhere no can I prove/disprove it.
Given two matrices $U \in \mathbb{R}^{m \times n},Z \in \mathbb{R}^{m \times n}$,, I can always write:
$U=\Delta Z + \Lambda$,  where $\Delta$ is a rotation matrix (orthogonal i.e $\Delta ^T \Delta=I$), and columns of Z are orthogonal to columns of $\Lambda$ and vice versa, i.e. $\Lambda^TZ = 0,$ and $Z^T\Lambda=0$.
This looks like extension of the idea of vector projections to matrix, but I neither get the intiution, not a mathematical proof of such an existence.
 A: This is not always possible. For convenience I write $R$ in place of $\Delta$ and $L$ in place of $\Lambda$. The system of equations, $U=RZ+L$ and $Z^TL=0$ reduces to
$$Z^TU=Z^TRZ.\tag{1}$$
So, if $\|Z^TU\|>\|Z\|^2$, where $\|\cdot\|$ denotes the operator norm (i.e. the largest singular value of a matrix), there certainly are no solutions for $R$.
But we can do more. Let $Z=P\pmatrix{S&0\\ 0&0_{(m-k)\times (n-k)}}Q^T$ be a singular value decomposition, where $S$ is an invertible diagonal matrix of rank $k\le\min(m,n)$. We can then rewrite equation $(1)$ as
$$
\pmatrix{I_k&0\\ 0&0_{(n-k)\times (m-k)}}PUQ=\pmatrix{I_k\\ 0&0_{(n-k)\times (m-k)}}P^TRP\pmatrix{S&0\\ 0&0_{(m-k)\times (n-k)}}.
$$
In other words, if you partition $PUQ$ into the form of $\pmatrix{A&B\\ C&D}$ where $A$ is $k\times k$, then a solution for $R$ exists if and only if $B=0$ and $AS^{-1}$ is the leading principal $k\times k$ of a real orthogonal matrix ($P^TRP$). Applying singular value decomposition on $AS^{-1}$, it can be shown that the previous necessary and sufficient condition is equivalent to $B=0$ and $I_k-(AS^{-1})^T(AS^{-1})$ is a positive semidefinite matrix of rank at most $\min(k,n-k)$.
