0
$\begingroup$

I was looking at the derivation of the normal equation from here.

Now, the author has used the fact that $A^{\mathrm T} B = B^{\mathrm T} A$ to reach the step shown in the below image. Can anyone provide some information like, when is it true, or how we can prove it?

$$J(\theta) = ((X\theta)^{\mathrm T} -y^{\mathrm T})(X\theta -y)$$ $$J(\theta) = (X\theta)^{\mathrm T} X\theta -\color{blue}{(X\theta)^{\mathrm T} y \color{black}{-} y^{\mathrm T} (X\theta)} +y^{\mathrm T} y$$ Note that $X\theta$ is a vector, and so is $y$. So when we multiply one by another, it doesn't matter what the order is (as long as the dimensions work out). So we can further simplify: $$J(\theta) = \theta^{\mathrm T} X^{\mathrm T} X \theta -\color{blue}{2(X\theta)^{\mathrm T} y} +y^{\mathrm T} y$$

$\endgroup$
  • 1
    $\begingroup$ $(AB)^T=B^TA^T$ $\endgroup$ – Botond Apr 25 '18 at 13:51
2
$\begingroup$

Note that $(A^{\rm T} B)^{\rm T} = B^{\rm T} (A^{\rm T})^{\rm T} = B^{\rm T} A$, so if you are constraining $A^{\rm T} B = B^{\rm T} A$, that implies $(A^{\rm T} B)^{\rm T} = A^{\rm T} B$, meanining $A^{\rm T} B$ is symmetric.

$\endgroup$
  • $\begingroup$ Got the symmetric part...So does it mean that $(X\theta)^Ty$ will always be a symmetric matrix?...how to prove that w.r.t normal equation derivation? $\endgroup$ – Sourajit Apr 25 '18 at 14:30
  • 1
    $\begingroup$ Is $X$ a matrix and both $\theta$ and $y$ are column vectors? If so, then $(X\theta)^{\rm T} y$ will always be symmetric because $X\theta$ is a vector and $(X\theta)^{\rm T} y$ is a scalar (a $1\times 1$ matrix), and scalars are always symmetric. $\endgroup$ – Rócherz Apr 25 '18 at 14:35
  • $\begingroup$ Yeah..got it..thanks ..you can check details about it Here.Although the equation given is wrong(there should be (+)ve sign before $y^Ty$),otherwise everything is correct. $\endgroup$ – Sourajit Apr 25 '18 at 14:42
0
$\begingroup$

Let's call $$C=AB$$ Then we have that $$C_{ij}=A_{ik}B_{kj}$$ We know that $$(M^T)_{ab}=M_{ba}$$ So we have that: $$(C^T)_{ij}=C_{ji}$$ $$(C^T)_{ij}=A_{jk}B_{ki}$$ $$(C^T)_{ij}=B_{ki}A_{jk}$$ $$(C^T)_{ij}=(B^T)_{ik}(A^T)_{kj}$$ And finally: $$C^T=B^TA^T$$ $$(AB)^T=B^TA^T$$

$\endgroup$
  • $\begingroup$ But here, $((X\theta)^Ty)^T=y^T(X\theta)$ while they have used $((X\theta)^Ty)=y^T(X\theta)$ $\endgroup$ – Sourajit Apr 25 '18 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.