7
$\begingroup$

How can I prove that $$ \frac 1 {n-1} \sum_{i=1}^n (X_i - \bar X)(Y_i-\bar Y) $$ is an unbiased estimate of the covariance $\operatorname{Cov}(X, Y)$ where $\bar X = \dfrac 1 n \sum_{i=1}^n X_i$ and $\bar Y = \dfrac 1 n \sum_{i=1}^n Y_i$ and $(X_1, Y_1), \ldots ,(X_n, Y_n)$ an independent sample from random vector $(X, Y)$?

$\endgroup$
  • 1
    $\begingroup$ Probably the reason why someone down-voted this question and someone voted to close it is that questions posted here should not be phrased in language suitable for assigning homework. $\endgroup$ – Michael Hardy Nov 18 '16 at 0:09
  • $\begingroup$ Do the multiplication, and deal with expectations of the resulting terms. $\endgroup$ – BruceET Nov 18 '16 at 0:58
  • 2
    $\begingroup$ One cannot show that it is an "unbiased estimate of the covariance". Perhaps you intend: unbiased estimator of the covariance $\endgroup$ – wolfies Nov 18 '16 at 3:26
  • $\begingroup$ @BruceET : Would you do something substantially different from what is in my answer posted below? $\qquad$ $\endgroup$ – Michael Hardy Nov 18 '16 at 23:04
  • $\begingroup$ That seems to work nicely. Proof that $E(S^2) = \sigma^2$ is similar, but easier. Perhaps my clue was too simplistic (omitting the $-\mu + \mu = 0$ trick). $\endgroup$ – BruceET Nov 18 '16 at 23:20
5
$\begingroup$

Let $\mu=\operatorname{E}(X)$ and $\nu = \operatorname{E}(Y).$ Then \begin{align} & \sum_{i=1}^n (X_i - \bar X)(Y_i-\bar Y) \\[10pt] = {} & \sum_{i=1}^n \Big( (X_i - \mu) + (\mu - \bar X)\Big) \Big((Y_i - \nu) + (\nu - \bar Y)\Big) \\[10pt] = {} & \left( \sum_i (X_i-\mu)(Y_i-\nu) \right) + \left( \sum_i (X_i-\mu)(\nu - \bar Y) \right) \\ & {} +\left( \sum_i (\mu-\bar X)(Y_i - \nu) \right) + \left( \sum_i(\mu-\bar X)(\nu - \bar Y) \right). \end{align}

The expected value of the first of the four terms above is $$ \sum_i \operatorname{E}\big( (X_i-\mu)(Y_i-\mu) \big) = \sum_i \operatorname{cov}(X_i,Y_i) = n\operatorname{cov}(X,Y). $$ The expected value of the second term is \begin{align} & \sum_i -\operatorname{cov}(X_i, \bar Y) = \sum_i - \operatorname{cov}\left(X_i, \frac {Y_1+\cdots+Y_n} n \right) \\[10pt] = {} & -n\operatorname{cov}\left( X_1, \frac{Y_1+\cdots+Y_n} n \right) = - \operatorname{cov}(X_1, Y_1+\cdots +Y_n) \\[10pt] & = -\operatorname{cov}(X_1,Y_1) + 0 + \cdots + 0 = -\operatorname{cov}(X,Y). \end{align} The third term is similarly that same number.

The fourth term is \begin{align} & \sum_i \overbrace{\operatorname{cov}(\bar X,\bar Y)}^{\text{No “} i \text{'' appears here.}} = n \operatorname{cov}(\bar X, \bar Y) = n \operatorname{cov}\left( \frac 1 n \sum_i X_i, \frac 1 n \sum_i Y_i \right) \\[10pt] = {} & n \cdot \frac 1 {n^2} \Big( \, \underbrace{\cdots + \operatorname{cov}(X_i, Y_j) + \cdots}_{n^2\text{ terms}} \, \Big). \end{align} This last sum is over all pairs of indices $i$ and $j$. But the covariances are $0$ except the ones in which $i=j$. Hence there are just $n$ nonzero terms, and we have $$ n\cdot \frac 1 {n^2} \left( \sum_i \operatorname{cov} (X_i,Y_i) \right) = n\cdot \frac 1 {n^2} \cdot n \operatorname{cov}(X,Y) = \operatorname{cov}(X,Y). $$

I leave the rest as an exercise.

$\endgroup$
6
$\begingroup$

Additional Comment, after some thought, following an exchange of Comments with @MichaelHardy:

His answer closely parallels the usual demonstration that $E(S^2) = \sigma^2$ and is easy to follow. However, the proof below, in abbreviated notation I hope is not too cryptic, may be more direct.

$$(n-1)S_{xy} = \sum(X_i-\bar X)(Y_i - \bar Y) = \sum X_i Y_i -n\bar X \bar Y = \sum X_i Y_i - \frac{1}{n}\sum X_i \sum Y_i.$$

Hence,

$$(n-1)E(S_{xy}) = E\left(\sum X_i Y_i\right) - \frac{1}{n}E\left(\sum X_i \sum Y_i\right)\\ = n\mu_{xy} - \frac{1}{n}[n\mu_{xy} + n(n-1)\mu_x \mu_y]\\ = (n-1)[\mu_{xy}-\mu_x\mu_y] = (n-1)\sigma_{xy},$$

So the expectation of the sample covariance $S_{xy}$ is the population covariance $\sigma_{xy} = \operatorname{Cov}(X,Y),$ as claimed.

Note that $\operatorname{E}(\sum X_i \sum Y_i)$ has $n^2$ terms, among which $\operatorname{E}(X_iY_i) = \mu_{xy}$ and $\operatorname{E}(X_iY_j) = \mu_x\mu_y.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.