Express this sum as a matrix equation Suppose I have matrices $G,H\in \mathbb{R}^{N\times K}$ and vector $w\in\mathbb{R}^K$. I want to express
$$
\sum_{i=1}^N \sum_{j=1}^N \sum_{k=1}^K w_k g_{ik}^2 h_{jk}^2
$$
as a matrix equation as cleanly as possible.
I am trying to do it in terms of trace and ordinary matrix multiplication, avoiding vectors of ones and Hadamard products but don't know if that's possible.
 A: Let $\mathrm g_k$ and $\mathrm h_k$ be the $k$-th columns of $\mathrm G$ and $\mathrm H$, respectively. Hence,
$$\begin{array}{rl} \displaystyle\sum_{i=1}^N \sum_{j=1}^N \sum_{k=1}^K w_k \, g_{ik}^2 \, h_{jk}^2 &= \displaystyle\sum_{k=1}^K w_k \left(\sum_{i=1}^N g_{ik}^2\right) \left(\sum_{j=1}^N h_{jk}^2\right)\\ &= \displaystyle\sum_{k=1}^K w_k \, \|\mathrm g_k\|_2^2 \, \|\mathrm h_k\|_2^2\\ &= \begin{bmatrix} \|\mathrm g_1\|_2^2\\ \|\mathrm g_2\|_2^2\\ \vdots\\ \|\mathrm g_K\|_2^2\end{bmatrix}^T \begin{bmatrix} w_1 & 0 & \dots & 0\\ 0 & w_2 & \dots & 0\\ \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & \dots & w_K\\\end{bmatrix}\begin{bmatrix} \|\mathrm h_1\|_2^2\\ \|\mathrm h_2\|_2^2\\ \vdots\\ \|\mathrm h_K\|_2^2\end{bmatrix}\\\\ &= (\mbox{diag}^{-1} (\mathrm G^T \mathrm G))^T \, \mbox{diag} (\mathrm w) \, \mbox{diag}^{-1} (\mathrm H^T \mathrm H)\end{array}$$
Let $\mathrm e_i$ be the $i$-th vector of the standard basis, and let $\mathrm E_i := \mathrm e_i \mathrm e_i^T$. Given an $n \times n$ matrix $\mathrm A$,
$$\mbox{diag}^{-1} (\mathrm A) = \sum_{i=1}^n a_{ii} \mathrm e_i = \sum_{i=1}^n (\mathrm e_i^T \mathrm A \mathrm e_i) \, \mathrm e_i = \sum_{i=1}^n \mathrm e_i \, (\mathrm e_i^T \mathrm A \mathrm e_i) = \sum_{i=1}^n \mathrm E_i \mathrm A \mathrm e_i = \tilde{\mathrm{E}} \, \mbox{vec} (\mathrm A)$$
where $\tilde{\mathrm{E}} := \begin{bmatrix} \mathrm E_1 & \mathrm E_2 & \cdots & \mathrm E_n\end{bmatrix}$. Thus,
$$\displaystyle\sum_{i=1}^N \sum_{j=1}^N \sum_{k=1}^K w_k \, g_{ik}^2 \, h_{jk}^2 = \mbox{vec}^T (\mathrm G^T \mathrm G) \, (\tilde{\mathrm{E}}^T \mbox{diag} (\mathrm w) \tilde{\mathrm{E}}) \, \mbox{vec} (\mathrm H^T \mathrm H)$$
