2
$\begingroup$

Assume that $(X_1,Y_1),...,(X_n,Y_n)$ is a sample on a two-dimensional random variable $(X,Y)$ and that $E[X^2], \ E[Y^2]$ and $E[XY]$ are all finite so that the variances and covariance are well-defined. Show that

$$S=\frac{1}{n-1}\sum_{i=1}^n(X_iY_i-\overline{X}\overline{Y})$$ is an unbiased estimator of $\text{Cov}[X,Y].$

So I'm supposed to show that $E[S]=\text{Cov[X,Y]}.$ Now

$$E\left[\sum_{i=1}^n(X_iY_i-\overline{X}\overline{Y})\right]=\sum_{i=1}^nE[X_iY_i]-nE[\overline{X}\overline{Y}]=n(E[XY]-E[\overline{X}\overline{Y}]).\tag1$$

According to the formula for variance the last step in $(1)$ can be rewritten as

$$n(\mu_x\mu_y+\text{Cov}[X,Y]-(\mu_x\mu_y+\text{Cov}[\overline{X}\overline{Y}]))=\text{Cov}[X,Y]-\text{Cov}[\overline{X}\overline{Y}])), \tag2$$

where $\mu_x$ and $\mu_y$ are the respective mean.

Questions:

  1. In $(2)$, I understand that from the covariance formula we get $$\mu_x\mu_y=E[X]E[Y]=E[XY]-\text{Cov}[X,Y],$$ but why is it also equal to $$\mu_x\mu_y=E[X]E[Y]=E[XY]-\text{Cov}[\overline{X}\overline{Y}]?$$ Shouldn't the sample means with overline cause any difference?

  2. How do I proceeed from here?

$\endgroup$
  • 1
    $\begingroup$ The overline quantities are defined as summations divided by n. Write them out, and expand the product of the averages over X and Y as a double summation and spit that double summation into a diagonal part where the indices are the same and a non-diagonal part where you sum over values for the indices that are different. $\endgroup$ – Count Iblis Aug 14 '18 at 22:16
1
$\begingroup$

For the first part, see that $E(\overline X)=\mu_x$ and $E(\overline Y)=\mu_y$ because: $$ E(\overline X)=E(\frac 1n\sum_i X_i)=\frac 1n\sum E(X_i)=\frac 1n (n\mu_x)=\mu_x. $$ For the second question, the solutions follows as: $$ \text{Cov}[\overline X\overline Y]=E[(\overline X-\mu_x)(\overline Y-\mu y)]=E(\overline X\overline Y)-\mu_x\mu_y. $$ Now see that $$ E(\overline X\overline Y)=\frac{1}{n^2}\sum_{i,j}E(X_iY_j)=\frac 1n E(XY)+\frac{n-1}{n}\mu_x\mu_y. $$ Hence: $$ \text{Cov}[\overline X\overline Y]=\frac 1n E(XY)-\frac 1n \mu_x\mu_y=\frac 1n \text{Cov}(XY). $$ Plug this in and you have: $$ E(S)=\frac{1}{n-1}\times n\times (\text{Cov}(XY)-\frac 1n \text{Cov}(XY))=\text{Cov}(XY). $$

$\endgroup$
  • $\begingroup$ Where does this follow from $$E[\overline X\overline Y)=\frac{1}{n^2}\sum_{i,j}E(X_iY_j] ?$$ Is it generally true that $$E[\overline{X_1}...\overline{X_k}] = \frac{1}{n^k}\sum_{i,j}E[X_iX_j]$$ $\endgroup$ – Parseval Aug 15 '18 at 9:24
  • 1
    $\begingroup$ Just multiply $\overline X=\frac 1n\sum_{i=1}^n X_i$ and $\overline Y=\frac 1n\sum_{j=1}^n Y_j$. $\endgroup$ – Arash Aug 15 '18 at 9:26
  • $\begingroup$ Ahh... okay I see then it is generally true i believe. Thanks! $\endgroup$ – Parseval Aug 15 '18 at 9:27
  • $\begingroup$ Could you please elaborate the last equality here: $$ E(\overline X\overline Y)=\frac{1}{n^2}\sum_{i,j}E(X_iY_j)=\frac 1n E(XY)+\frac{n-1}{n}\mu_x\mu_y. $$ $\endgroup$ – Parseval Aug 15 '18 at 13:40
  • $\begingroup$ If $i=j$, then $E(X_iY_j)=E(X_iY_i)=E(XY)$ and there are $n$ different pairs as such; if $i\neq j$ then $X_i$ and $Y_j$ are independent therefore $E(X_iY_j)=\mu_x\mu_y$ and there are $\binom{n}{2}$ of such pairs. $\endgroup$ – Arash Aug 15 '18 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.