Little notation question about matrix multiplications / quadratic forms I'm a bit confused about expanding out the notation of product of matrices, in the context of quadratic forms.
If $x \in \mathbb{R}^n, \, \, A \in \mathbb{R}^{n \times n}$ then
$$x^TAx = \sum_{i,j=1}^na_{ij}x_ix_j$$
But then if I consider a matrix $X \in \mathbb{R}^{n \times n}$ how should I write the expanded form of
$$X^TAX = \, \,...\, \, ?$$
This time the result will be a matrix.. will it be something like
$$X^TAX  = \sum_{i,j=1}^n a_{ij}x_ix_j^T$$
and if yes, why?
Sorry if this is pretty straightforward but it always happens to get a little bit stuck with matrix notation.
Many thanks,
James
 A: It's easy to see why your attempt is wrong by considering a matrix $X$ of dimension $n\times p$. Then $X^TAX$ has dimension $p\times p$, while your formula yields a $n\times n$ matrix.
You can work out the contribution of $a_{i,j}$ to the product by writing the product:
$$X^TAX=\sum_{i,j,k,l} x_{i,k}a_{i,j}x_{j,l}e_{k,l}$$
Where $e_{k,l}$ are elementary matrices, and the sum runs on all valid indices. All elements of the matrix $e_{k,l}$ are $0$ except at index $(k,l)$ where it's $1$.
Then:
$$X^TAX=\sum_{i,j} a_{i,j} \left(\sum_{k,l} x_{i,k}x_{j,l}e_{k,l}\right)$$
And inside the parentheses we recognize $x_i^Tx_j$, where $x_i$ is the $i$th row.
If it's difficult to recognize, consider this: the element at index $(k,l)$ is $x_{i,k}x_{j,l}$. That is, the matrix $\sum_{k,l} x_{i,k}x_{j,l}e_{k,l}$ is an outer product $uv^T$ where $u$ and $v$ are column vectors. The $k$th element of $u$ must be $x_{i,k}$, that is $u$ has the same elements as the row $x_i$, except it's a column vector, so $u=x_i^T$.
And the $l$th element of $v$ is $x_{j,l}$, and in the product you get $v^T$, which is a row vector, hence $v^T=x_j$. Therefore $\sum_{k,l} x_{i,k}x_{j,l}e_{k,l}=x_i^Tx_j$.
A: The $(i,j)$-coefficient of the matrix $X^TAX$ is given by
\begin{equation}
\sum_{k,l=1}^nx_{ki}a_{kl}x_{lj},
\end{equation}
where $x_{ij}$ (resp. $a_{ij}$) is the coefficient of $X$ (resp. $A$).
