# Orthogonalization of a set of random vectors

Suppose $w_1$ and $w_2$ are zero-mean jointly Gaussian random vectors. Further suppose that they have a covariance matrix given by $$\mathbf{cov}\begin{bmatrix}w_1 \\ w_2\end{bmatrix} = \begin{bmatrix}I & X \\ X^T & I\end{bmatrix}$$ Let $X = \begin{bmatrix}U_1 & \bar U_1\end{bmatrix} \begin{bmatrix}\Sigma & 0 \\ 0 & 0\end{bmatrix} \begin{bmatrix}U_2 & \bar U_2\end{bmatrix}^T$ be the full SVD of $X$. Now decompose $w_i$ as follows: \begin{align*} w_1 &= U_1 \eta_1 + \bar U_1 \nu_1 \\ w_2 &= U_2 \eta_2 + \bar U_2 \nu_2 \end{align*} This is done by projecting, $\eta_1 = U_1^T w_1$, etc. It's easy to verify that the random vectors $\begin{bmatrix}\eta_1 \\\eta_2\end{bmatrix}$, $\nu_1$, $\nu_2$ are mutually independent. Furthermore, they have covariances: \begin{align*} \mathbf{cov}\begin{bmatrix}\eta_1 \\ \eta_2\end{bmatrix} &= \begin{bmatrix}I & \Sigma \\ \Sigma & I\end{bmatrix}, & \mathbf{cov}\nu_1 &= I, & \mathbf{cov}\nu_2 &= I \end{align*} In a sense, this decomposition provides a new set of coordinates such that the associated covariance matrices now have diagonal blocks. I am looking for a similar result when I have more than two vectors. Say we have $w_1$, $w_2$, and $w_3$ jointly Gaussian, now with $$\mathbf{cov}\begin{bmatrix}w_1 \\ w_2 \\ w_3\end{bmatrix} = \begin{bmatrix}I & X & Y\\ X^T & I & Z \\ Y^T & Z^T & I\end{bmatrix}$$ I have tried to extend the same technique as in the two-vector case, but run into problems because coordinates that "diagonalize" $w_1$ and $w_2$ don't also diagonalize with respect to $w_3$. I suspect one might need to take into account every possible subset of the vectors... This leads me to conjecture a decomposition of the form: \begin{align*} w_1 &= M_1 \eta_1 + P_1 \alpha_1 + Q_1 \beta_1 \phantom{+R_1\alpha_3}+ \bar U_1 \nu_1 \\ w_2 &= M_2 \eta_2 + P_2 \alpha_2 \phantom{+Q_2\alpha_3}+ R_2 \gamma_2 + \bar U_2 \nu_2 \\ w_3 &= M_3 \eta_3 \phantom{+P_3\alpha_3} + Q_3 \beta_3 + R_3\gamma_3 + \bar U_3 \nu_3 \end{align*} where $\begin{bmatrix}M_1 & P_1 & Q_1 & \bar U_1\end{bmatrix}$ is orthogonal and similarly for the other two equations, and where $\begin{bmatrix}\eta_1 \\\eta_2\\\eta_3\end{bmatrix}$, $\begin{bmatrix}\alpha_1 \\\alpha_2\end{bmatrix}$, $\begin{bmatrix}\beta_1 \\\beta_3\end{bmatrix}$, $\begin{bmatrix}\gamma_2 \\\gamma_3\end{bmatrix}$, $\nu_1$, $\nu_2$, $\nu_3$ are mutually independent, again with covariances with diagonal blocks, as in the two-vector case.

Any thoughts on how I might go about proving whether such a decomposition exists for $n\ge 3$, and computing it?

-