# When can we say that $A^{\mathrm T} B = B^{\mathrm T} A$?

I was looking at the derivation of the normal equation from here.

Now, the author has used the fact that $$A^{\mathrm T} B = B^{\mathrm T} A$$ to reach the step shown in the below image. Can anyone provide some information like, when is it true, or how we can prove it?

$$J(\theta) = ((X\theta)^{\mathrm T} -y^{\mathrm T})(X\theta -y)$$ $$J(\theta) = (X\theta)^{\mathrm T} X\theta -\color{blue}{(X\theta)^{\mathrm T} y \color{black}{-} y^{\mathrm T} (X\theta)} +y^{\mathrm T} y$$ Note that $$X\theta$$ is a vector, and so is $$y$$. So when we multiply one by another, it doesn't matter what the order is (as long as the dimensions work out). So we can further simplify: $$J(\theta) = \theta^{\mathrm T} X^{\mathrm T} X \theta -\color{blue}{2(X\theta)^{\mathrm T} y} +y^{\mathrm T} y$$

• $(AB)^T=B^TA^T$ – Botond Apr 25 '18 at 13:51

Note that $(A^{\rm T} B)^{\rm T} = B^{\rm T} (A^{\rm T})^{\rm T} = B^{\rm T} A$, so if you are constraining $A^{\rm T} B = B^{\rm T} A$, that implies $(A^{\rm T} B)^{\rm T} = A^{\rm T} B$, meanining $A^{\rm T} B$ is symmetric.
• Got the symmetric part...So does it mean that $(X\theta)^Ty$ will always be a symmetric matrix?...how to prove that w.r.t normal equation derivation? – Sourajit Apr 25 '18 at 14:30
• Is $X$ a matrix and both $\theta$ and $y$ are column vectors? If so, then $(X\theta)^{\rm T} y$ will always be symmetric because $X\theta$ is a vector and $(X\theta)^{\rm T} y$ is a scalar (a $1\times 1$ matrix), and scalars are always symmetric. – Rócherz Apr 25 '18 at 14:35
• Yeah..got it..thanks ..you can check details about it Here.Although the equation given is wrong(there should be (+)ve sign before $y^Ty$),otherwise everything is correct. – Sourajit Apr 25 '18 at 14:42
Let's call $$C=AB$$ Then we have that $$C_{ij}=A_{ik}B_{kj}$$ We know that $$(M^T)_{ab}=M_{ba}$$ So we have that: $$(C^T)_{ij}=C_{ji}$$ $$(C^T)_{ij}=A_{jk}B_{ki}$$ $$(C^T)_{ij}=B_{ki}A_{jk}$$ $$(C^T)_{ij}=(B^T)_{ik}(A^T)_{kj}$$ And finally: $$C^T=B^TA^T$$ $$(AB)^T=B^TA^T$$
• But here, $((X\theta)^Ty)^T=y^T(X\theta)$ while they have used $((X\theta)^Ty)=y^T(X\theta)$ – Sourajit Apr 25 '18 at 14:21