How do you prove that $tr(B^{T} A )$ is a inner product?

Question

Consider the vectorspace of all real $m \times n$ vectors and define an inner product $\langle A,B\rangle = \operatorname{tr}(B^T A)$. "tr" stands for "trace" which is the sum of the diagonal entries of a matrix.

How do you prove that $\operatorname{tr}(B^T A)$ is indeed a inner product?

Kind regards

Answer 1

For every $A=(A_{ij}) \in \mathbb{R}^{m\times n}$ we have $$ \langle A,A\rangle=\text{tr}(A^TA)=\sum_{i=1}^n(A^TA)_{ii}=\sum_{i=1}^n\sum_{j=1}^mA^T_{ij}A_{ji}=\sum_{i=1}^m\sum_{j=1}^nA_{ij}^2 \ge 0, $$ and $$ \langle A,A\rangle=\sum_{i=1}^m\sum_{j=1}^nA_{ij}^2 = 0\iff (A_{ij}=0 \quad \forall i,j) \iff A=0 $$ Since $$ \text{tr}(X^T)=\text{tr}(X), \quad \text{tr}(X+Y)=\text{tr}(X)+\text{tr}(Y), \quad \text{tr}(\lambda X)=\lambda\text{tr}(X) $$ for every $X,Y \in \mathbb{R}^{n\times n}$, and $\lambda \in \mathbb{R}$, therefore, for every $A,B, C \in \mathbb{R}^{m\times n}$, and $\lambda \in \mathbb{R}$ we have \begin{eqnarray} \langle A,B\rangle&=&\text{tr}(B^TA)=\text{tr}((B^TA)^T)=\text{tr}(A^TB)=\langle B,A\rangle,\\ \langle \lambda A+B,C\rangle&=&\text{tr}(C^T(\lambda A+B))=\text{tr}(\lambda C^TA+C^TB)=\lambda\text{tr}(C^TA)+\text{tr}(C^TB)\\ &=&\lambda\langle A,C\rangle+\langle B,C\rangle. \end{eqnarray}

Answer 2

If $A=(a_{ij})$ and $B=(b_{ij})$ and $C=B^TA=(c_{ij})$ then $$(c)_{ij}=\sum_{k=1}^m b_{ki}a_{kj}$$

$$\mathrm{tr}(B^TA)=\sum_{i=1}^n c_{ii}=\sum_{i=1}^n\sum_{k=1}^m b_{ki}a_{ki}$$ so we see that $\langle.,.\rangle$ is an inner product (Euclidian) by identifying $\mathcal M_{m\times n}(\mathbb R)$ to $\mathbb R^{m\times n}$.

Answer 3

You want to verify all the properties of a real inner product (since we're looking at a real vector space). Using your notation $\langle A,B \rangle = \mathrm{tr}(B^T A)$, we want to check that:

1. $\langle A,B \rangle = \langle B,A \rangle $ (symmetry)
2. $\langle cA + B, C \rangle = c \langle A,C \rangle + \langle B,C \rangle$ (linearity)
3. $\langle A, A \rangle > 0$ unless $A = 0$ (definiteness)

You'll need to use some properties about the trace in order to prove some of these conditions hold.

Answer 4

As it is explained here, we can use the vectorization operator on matrices $\boldsymbol A$ and $\boldsymbol B$ to rewrite the trace of the product as

$$\begin{align} \text{tr}\left(\boldsymbol A^T \boldsymbol B \right) &= \text{vec}\left(\boldsymbol B\right)^T \text{vec}\left(\boldsymbol A\right) \\ &= \left\langle\text{vec}\left(\boldsymbol A\right),\text{vec}\left(\boldsymbol B\right)\right\rangle \end{align}$$

which is the inner product for the vector case.