Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am looking to prove the following

Let $z$ be an $m\times$ 1 random vector with $E(z)=\mu$ and $\operatorname{Cov}(z)=V$ and let $A$ be an $m\times m$ non-stochastic matrix. Then the following identity holds true:

$$E(z'Az) = \operatorname{trace}(AV) + \mu'A\mu$$

I am failing to find a straight forward proof to it.

p.s: This is not a homework assignment (although something in my homework DOES rely on this theorem, and I would love to know why it holds).

I reduced the problem to the following: If we call z'=X and z'A'=Y then we already know that: $\operatorname{cov}(X,Y) = E(XY')-E(X)E(Y)' \rightarrow E(XY')=\operatorname{cov}(X,Y)+E(X)E(Y)' \rightarrow $ $E(z'Az)=\operatorname{cov}(z',z'A')+E(z')E(z'A')'=\operatorname{cov}(z',z'A')+\mu'(\mu'A')'$ $=\operatorname{cov}(z',z'A')+\mu'A\mu$

Which leaves us to show that $\operatorname{cov}(z',z'A')=\operatorname{trace}(AV)$

Can someone please help by showing how this is true?

Is there another direction to this proof that I am missing?

Thanks.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

It follows from the fact that $\operatorname{trace}(AB)=\operatorname{trace}(BA)$, i.e. one can cyclically permute matrices when taking a trace, it doesn't change the end result. Now, since $\operatorname{cov}(z',z'A')=(z-\mu)'(Az-A\mu)=(z-\mu)'A(z-\mu)$ is a number, $(z-\mu)'A(z-\mu)=\operatorname{trace}((z-\mu)'A(z-\mu))$ trivially. Then permute the $(z-\mu)'$ to the back to get $\operatorname{trace}(A(z-\mu)(z-\mu)')$. But the last two factors form just the covariance matrix of $z$ after taking the expectation of course.

share|improve this answer
    
Great answer - thank you! I keep being amazed at how simple these proofs are once we go into matrix algebra... –  Tal Galili Dec 25 '11 at 8:02
    
BTW, is it always true to say that E(trace[Az])=trace[A*E(z)] ? (or just if Az is a number?) –  Tal Galili Dec 25 '11 at 8:57
1  
It's true because $A$ is a non-stochastic matrix. And because $E$ can be shifted through any linear operations like taking a trace or multiplying by a non-stochastic matrix. By the way, $Az$ is not a number, it's a column random vector. But $z'Az$ is a single random variable. –  Raskolnikov Dec 25 '11 at 9:52
    
Thanks again Raskolnikov. It is a pleasure to learn all of these things (and amazing how few of these proofs are documented in the books I am reading). If you have any good suggestions to books with proofs - I will gladly search for it. With regards, Tal –  Tal Galili Dec 25 '11 at 10:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.