Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} \mathbf{X}_1^{\top} & \mathbf{X}_2^{\top} & \cdots & \mathbf{X}_N^{\top} \end{bmatrix} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix}^{\top} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\mathbf{X}_{\bullet}^{\top}\mathbf{X}_{\bullet} \end{align} where $\mathbf{X}_{\bullet}$ is an $NG \times K$ matrix whose $i^{th}$ row is $\mathbf{X}_i$.

I'd like to do something similar with this summation: \begin{equation} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i \end{equation} where $\mathbf{\Omega}$ is some arbitrary symmetric $G \times G$ matrix, say for illustration \begin{equation} \mathbf{\Omega} =\begin{bmatrix} g_{11} & g_{12} & \cdots & g_{1G} \\ g_{12} & g_{22} & \cdots & g_{2G} \\ \vdots & \vdots & \ddots & \vdots \\ g_{1G} & g_{2G} & \cdots & g_{GG} \end{bmatrix} \end{equation}

This problem is not so simple as the first one, because now I have some $\mathbf{\Omega}^{-1}$ matrix in the middle of the sum. However, I think it can still be expressed in matrix notation.

For those curious, this matrix algebra is behind the theory of Generalized Least Squares. Thanks very much for your help!

Edit I think I have figured it out. I have posted an answer below -- please let me know if you think it's correct!


Let's try this. \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i&=\mathbf{X}_1\mathbf{\Omega}^{-1}\mathbf{X}_1+\mathbf{X}_2\mathbf{\Omega}^{-1}\mathbf{X}_2+\cdots+\mathbf{X}_N\mathbf{\Omega}^{-1}\mathbf{X}_N \\ &=\begin{bmatrix} \mathbf{X}_1^{\top} & \mathbf{X}_2^{\top} & \cdots & \mathbf{X}_N^{\top} \end{bmatrix} \begin{bmatrix} \mathbf{\Omega}^{-1}\mathbf{X}_1 \\ \mathbf{\Omega}^{-1}\mathbf{X}_2 \\ \vdots \\ \mathbf{\Omega}^{-1}\mathbf{X}_N \end{bmatrix} \\ &=\begin{bmatrix} \mathbf{X}_1^{\top} & \mathbf{X}_2^{\top} & \cdots & \mathbf{X}_N^{\top} \end{bmatrix} \begin{bmatrix} \mathbf{\Omega}^{-1} & \mathbf{0} & \cdots & \mathbf{0} \\ \mathbf{0} & \mathbf{\Omega}^{-1} & \cdots & \mathbf{0} \\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{0} & \mathbf{0} & \cdots & \mathbf{\Omega}^{-1} \end{bmatrix} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix}^{\top} \begin{bmatrix} \mathbf{\Omega}^{-1} & \mathbf{0} & \cdots & \mathbf{0} \\ \mathbf{0} & \mathbf{\Omega}^{-1} & \cdots & \mathbf{0} \\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{0} & \mathbf{0} & \cdots & \mathbf{\Omega}^{-1} \end{bmatrix} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\mathbf{X}_\bullet^{\top}\left(\mathbf{I}_N \otimes \mathbf{\Omega}^{-1}\right)\mathbf{X}_\bullet \end{align} where "$\otimes$" denotes the Kronecker Product.


Put $\mathbf{X}_i$'s next to each other to form a matrix. Oh, in fact you did. Yes, it is $X$.

Notice that $$\mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$$ is the $i$'th element on the diagonal of


So, since you're summing these up, $$\begin{equation} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i \end{equation} = trace(\mathbf{X}^{\top}\mathbf{\Omega}^{-1}\mathbf{X}).$$ Done!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.