Consider the vectorspace of all real $m \times n$ vectors and define an inner product $\langle A,B\rangle = \operatorname{tr}(B^T A)$. "tr" stands for "trace" which is the sum of the diagonal entries of a matrix.

How do you prove that $\operatorname{tr}(B^T A)$ is indeed a inner product?

Kind regards

  • 3
    $\begingroup$ Well, which properties are needed? Which do you have trouble with? $\endgroup$ Aug 26 '13 at 20:31
  • 10
    $\begingroup$ For intuition, note that $tr(B^T A)$ is what you'd get if you reshaped $A$ and $B$ into column vectors and took the standard inner product. $\endgroup$
    – littleO
    Aug 26 '13 at 20:58
  • 3
    $\begingroup$ It is an inner product on $M_n(R)$ I believe, but not $M_n(C)$. $\endgroup$
    – michek
    May 1 '14 at 14:42

For every $A=(A_{ij}) \in \mathbb{R}^{m\times n}$ we have $$ \langle A,A\rangle=\text{tr}(A^TA)=\sum_{i=1}^n(A^TA)_{ii}=\sum_{i=1}^n\sum_{j=1}^mA^T_{ij}A_{ji}=\sum_{i=1}^m\sum_{j=1}^nA_{ij}^2 \ge 0, $$ and $$ \langle A,A\rangle=\sum_{i=1}^m\sum_{j=1}^nA_{ij}^2 = 0\iff (A_{ij}=0 \quad \forall i,j) \iff A=0 $$ Since $$ \text{tr}(X^T)=\text{tr}(X), \quad \text{tr}(X+Y)=\text{tr}(X)+\text{tr}(Y), \quad \text{tr}(\lambda X)=\lambda\text{tr}(X) $$ for every $X,Y \in \mathbb{R}^{n\times n}$, and $\lambda \in \mathbb{R}$, therefore, for every $A,B, C \in \mathbb{R}^{m\times n}$, and $\lambda \in \mathbb{R}$ we have \begin{eqnarray} \langle A,B\rangle&=&\text{tr}(B^TA)=\text{tr}((B^TA)^T)=\text{tr}(A^TB)=\langle B,A\rangle,\\ \langle \lambda A+B,C\rangle&=&\text{tr}(C^T(\lambda A+B))=\text{tr}(\lambda C^TA+C^TB)=\lambda\text{tr}(C^TA)+\text{tr}(C^TB)\\ &=&\lambda\langle A,C\rangle+\langle B,C\rangle. \end{eqnarray}

  • 2
    $\begingroup$ what does this imply about the relationship between $\text{tr}(A^TB) \text{and } \text{tr}(AB^T)$? $\endgroup$ May 3 '15 at 0:00
  • 4
    $\begingroup$ @BenjaminLoya Your question is already answered in there! $\endgroup$
    – Mercy King
    May 3 '15 at 0:24

If $A=(a_{ij})$ and $B=(b_{ij})$ and $C=B^TA=(c_{ij})$ then $$(c)_{ij}=\sum_{k=1}^m b_{ki}a_{kj}$$

$$\mathrm{tr}(B^TA)=\sum_{i=1}^n c_{ii}=\sum_{i=1}^n\sum_{k=1}^m b_{ki}a_{ki}$$ so we see that $\langle.,.\rangle$ is an inner product (Euclidian) by identifying $\mathcal M_{m\times n}(\mathbb R)$ to $\mathbb R^{m\times n}$.

  • $\begingroup$ Nice answer, although you should probably make the minor ajustments necessary to handle the "non-square matrix" case (which is the OP's original setting). $\endgroup$
    – M Turgeon
    Aug 26 '13 at 20:49
  • $\begingroup$ Done. Thanks for your comment. $\endgroup$
    – user63181
    Aug 26 '13 at 21:07
  • $\begingroup$ Maybe this can help, do the proof is in Spanish and proved for the complex case. $\endgroup$
    – Las Des
    May 2 at 4:39

You want to verify all the properties of a real inner product (since we're looking at a real vector space). Using your notation $\langle A,B \rangle = \mathrm{tr}(B^T A)$, we want to check that:

  1. $\langle A,B \rangle = \langle B,A \rangle $ (symmetry)
  2. $\langle cA + B, C \rangle = c \langle A,C \rangle + \langle B,C \rangle$ (linearity)
  3. $\langle A, A \rangle > 0$ unless $A = 0$ (definiteness)

You'll need to use some properties about the trace in order to prove some of these conditions hold.


As it is explained here, we can use the vectorization operator on matrices $\boldsymbol A$ and $\boldsymbol B$ to rewrite the trace of the product as

$$\begin{align} \text{tr}\left(\boldsymbol A^T \boldsymbol B \right) &= \text{vec}\left(\boldsymbol B\right)^T \text{vec}\left(\boldsymbol A\right) \\ &= \left\langle\text{vec}\left(\boldsymbol A\right),\text{vec}\left(\boldsymbol B\right)\right\rangle \end{align}$$

which is the inner product for the vector case.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.