# What are the meanings of matrix constructs of pre-multiplication of the matrix transpose and post-multiplication by the matrix itself.

In many books and articles I sometimes see similar matrix constructs, multiplication of the matrix transpose (or inverse), some other matrix, and then the first matrix itself, like this: $$M^T AM$$ or $$M^{-1} A M$$.

For square matrices $$M$$ I've found an example of definition of matrix congruence, that is, existence of certain matrices $$A,B,P$$ related so that $$P^T A P = B$$ names $$A$$ and $$B$$ congruent. However, this only applies to square $$A,B$$ and invertible $$P$$. Also I'm not aware of practical usage of this concept.

Again, for square $$A, B, P$$ there exists a concept of matrix similarity, that is, relation of $$B=P^{-1}AP$$ names $$A$$ and $$B$$ similar. I'm more familiar with this concept since it's frequently used in computer graphics when $$B$$ acts as "base" model-view matrix, and series of $$P_i$$ act as a set of modelling matrices for series of objects in a scene.

However, these "tricks" for non-square matrices are also frequently used in literature, for example, in normal equations for non-linear least squares methods and algorithms.

The question is: what are the informal explanations of $$M^T AM$$ or $$M^{-1} A M$$ matrix "tricks" and how they are used in applications?

• if you know a pinch of group theory, and e.g. A is invertible, then $M^{-1} A M$ is conjugating $A$. The congruence transform $M^T A M$ makes perfect sense in context of bilinear forms. Sylvester's Law of Interita and e.g. $LDL^T$ or even Cholesky factorization can be interpretted as applications of Sylvester's Law. May 31 '20 at 3:27

There’s no “trick” here. These all represent a change of basis (coordinate system) for various objects that the matrix $$A$$ might represent. The matrix $$M$$ converts coordinates of a vector from one basis to another.
When $$A$$ represents a linear transformation, the appropriate formula is $$M^{-1}AM$$. The matrix $$M$$ converts from some coordinate system into the one that $$A$$ is expecting, and $$M^{-1}$$ converts its output back into the original coordinate system. Algebraically, if we have $$x=Mx'$$ and $$y=My'$$, then from $$y=Ax$$ we have $$My'=AMx'$$, and multiplying both sides by $$M^{-1}$$ produces $$y'=(M^{-1}AM)x'$$.
Similarly, $$M^TAM$$ is a change of basis when $$A$$ represents a bilinear or quadratic form. Here, we’re working with an expression of the form $$x^TAy$$, from which $$x^TAy = (Mx')^TA(My) = x'^T(M^TAM)y$$. You might also see $$M^TAM$$ when $$A$$ represents a linear transformation, but that’s a special case of $$M^{-1}AM$$ when $$M^{-1}=M^T$$, which occurs when the change of basis is achieved via an orthogonal transformation.
• But what about the cases where $M^T A M$ is used with non-square $M$? In that case $A$ has to be square anyways to satisfy the multiplication rules, but $M$ doesn't seem to represent any meaningful change of basis. I've seen this in LSM method for over-determined set of points. May 31 '20 at 11:11
• @mbaitoff It all really depends on context—you have to know what the matrices represent. It could still be a change of basis, but the basis is for a subspace of the ambient space. Or, $A$ might be the Gram matrix of an inner product, in which case $M^TAM$ computes all of the pairwise inner products of the columns of $M$ “in bulk.”