I am looking at a paper which has the following manipulation:

$$ \exp \big(\sum_i(y_i - Ax_i)^T V^{-1}(y_i - Ax_i)\big) $$

Here $A$ is a matrix. $V$ is a symmetric, positive definite matrix (inverse of a covariance i.e. a precision matrix). The paper goes on to write this as:

$$ \exp\big(\operatorname{tr}(V^{-1}(Y-Ax)(Y-Ax)^T)\big) $$

However, it is not clear to me how this comes about! Can someone point me to what matrix identity has been used or guide to some proof of this?

  • $\begingroup$ What are $x_i$ and $y_i$, are they vectors? What about $Y$ and $x$? $\endgroup$ – Pedro M. Feb 28 '15 at 2:44
  • 2
    $\begingroup$ It's a very common trick: if $x=a'Sb$ is a scalar, then $x=\text{Tr}(x)=\text{Tr}(a'Sb)=\text{Tr}(Sba')$. The first equality uses the fact that $x$ is a scalar and the last equality uses $\text{Tr}(AB)=\text{Tr}(BA)$ for all $A,B$ such that the matrix products make sense. $\endgroup$ – Kim Jong Un Feb 28 '15 at 2:58

If $A\in\mathbb{R}^{n\times m}$ and $B\in\mathbb{R}^{m\times n}$ (and we do not assume $n=m$) then $\operatorname{trace}(AB)=\operatorname{trace}(BA)$.

First you have the scalar $$ \underbrace{(y_i - Ax_i)^T}\ \underbrace{V^{-1}(y_i - Ax_i)} $$ which is a product of matrices, but as a scalar it is its own trace. So it is equal to $$ \operatorname{trace}\left(\underbrace{V^{-1}(y_i - Ax_i)}\ \underbrace{(y_i - Ax_i)^T}\right). $$ Then the sum of the traces equals the trace of the sum, and the sum is $$ V^{-1}(Y-Ax)(Y-Ax)^T. $$

There is more on this in Wikipedia's article "Estimation of covariance matrices" (last time I looked).

  • $\begingroup$ Also thanks for the wikipedia entry. That is a very useful page. $\endgroup$ – Luca Feb 28 '15 at 12:10

If we stack the $x_i$s and $y_i$s column wise in a matrix, we get the matrix $X$ and $Y$ respectively.

And, $\sum_i(y_i - Ax_i)^TV^{-1}(y_i - Ax_i) = tr\left((Y - AX)^TV^{-1}(Y - AX)\right)$

Also, since the trace operator is invariant to cyclic permutations, $tr\left((Y - AX)^TV^{-1}(Y - AX)\right) = tr\left(V^{-1}(Y - AX)(Y - AX)^T\right)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.