Why is Covariance defined as:

  • $\operatorname{cov}(X,Y) = E[(X-E[X])(Y-E[Y])]$

My book simply states this identity but doesn't explain how it is derived. I know that covariance can also be written as:

  • $\operatorname{cov}(X,Y) = E[XY] -E[X]E[Y]$

I know how to derive this identity from the first definition.

  • 1
    $\begingroup$ Consider $\mathbb{E}[XY]$ and think of when $\mathbb{E}[XY]=\mathbb{E}[X] \mathbb{E}[Y]$ $\endgroup$ – Karl Oct 5 '15 at 5:26
  • $\begingroup$ Expectation is a Linear Operator in the sense $E[aX+bY]=aE[X]+bE[Y]$ $\endgroup$ – Ekaveera Kumar Sharma Oct 5 '15 at 6:51
  • 1
    $\begingroup$ @Karl Done. And then what? $\endgroup$ – Did Oct 5 '15 at 7:37
  • $\begingroup$ @Did I was thinking that the situation motivates the definition of covariance as $\mathbb{E}[XY]=\mathbb{E}[X] \mathbb{E}[Y]$ when $cov(X,Y) =0$ Is this incorrect? $\endgroup$ – Karl Oct 5 '15 at 7:57
  • $\begingroup$ @Karl More meaningless than incorrect. "the situation" Which situation? "the definition of covariance as 𝔼[XY]=𝔼[X]𝔼[Y]" 𝔼[XY]=𝔼[X]𝔼[Y] is not a definition of covariance. $\endgroup$ – Did Oct 5 '15 at 8:01

I don't know if there is any technical reason behind this identity. There is a simple intuition behind the identity. We want to know if the variables X and Y are linearly related or not.i.e, can we say something like Y= m X where m is a constant. If m is positive, then Y increases as X increases. If m is negative, the Y decreases as X increases. First we take rescale both the variables around their mean by looking at P=X-E[x] and Q=Y-E[Y]. If P and Q are both positive or negative, then it adds to the covariance. If one of them is positive and the other is negative ,it subtracts from the covariance. Finally, the expectation gives the average behaviour.

  • $\begingroup$ This was perfect - exactly what I needed to hear. Thank you! $\endgroup$ – David South Oct 5 '15 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.