4
$\begingroup$

Suppose that I have a Poisson distribution $P(\lambda)$.

Let $X_1,X_2..,X_n$ be independent random variables from the distribution mentioned above.

Let us define sample variance $S^2 = \frac{1}{n-1} \sum (X_i - M)^2 $ and sample mean as $M = \frac{1}{n}\sum X_i $.

I want to find the covariance, $Cov(M,S^2)$. I've seen from this answer that when the distribution is symmetric, they are uncorrelated, which makes it zero. I also know that for large $\lambda$ values Poisson distribution is very close to Gaussian distribution, thus becoming symmetric, and probably the covariance is close to zero.

However, I want to find the exact value of $Cov(M,S^2)$, as I am working with small values of $\lambda$


Currently I tried the following:

$Cov(M,S^2) =E( (M-\lambda) (S^2-\lambda) ) = E(MS^2)-\lambda E(M) - \lambda E(S^2) + \lambda^2$

Thus, $Cov(M,S^2) = E(MS^2) - \lambda^2 - \lambda^2 + \lambda^2 = E(MS^2) -\lambda^2$

I am having trouble with calculating $E(MS^2)$.

Any idea?

$\endgroup$
  • $\begingroup$ A square is missing in the definition of $S^2$, isn't it? $\endgroup$ – Davide Giraudo Nov 12 '12 at 13:00
  • $\begingroup$ @DavideGiraudo, you are right, I made a typo. Thanks for pointing that out. $\endgroup$ – Andrey Rubshtein Nov 12 '12 at 13:09
  • 1
    $\begingroup$ We can expand the square in the definition of $S^2$ to get a more simple expression, then express $E(MS^2)$ as a double sum. You will have to distinguish when the indexes are the same or not. $\endgroup$ – Davide Giraudo Nov 12 '12 at 13:13
5
$\begingroup$

Let us first center everything, using $\bar X_k=X_k-\lambda$ and $\bar M=M-\lambda$. Then $$ \mathrm{Cov}(M,S^2)=\mathbb E(\bar MS^2)=\mathbb E(\bar X_1S^2)=\frac1{n-1}\mathbb E(\bar X_1U), $$ where $$ U=\sum\limits_{k=1}^n(\bar X_k-\bar M)^2. $$ Note that $U$ is a linear combination of products $\bar X_k^2$ and $\bar X_k\bar X_i$ for $i\ne k$. Amongst these products, many will not contribute to the expectation of $\bar X_1U$ since $\mathbb E(\bar X_1\bar X_k\bar X_i)=0$ for every $k\ne i$ and $\mathbb E(\bar X_1\bar X_k^2)=0$ for every $k\ne1$.

Hence, one needs only the coefficient of $\bar X_1^2$ in $U$, which is $c_n=\left(\frac{n-1}n\right)^2+(n-1)\frac1{n^2}=\frac{n-1}n$. This yields $\mathbb E(\bar X_1U)=c_n\mathbb E(\bar X_1^3)$ and $\mathrm{Cov}(M,S^2)=\frac1{n-1}c_n\mathbb E(\bar X_1^3)=\frac1n\mathbb E(\bar X_1^3)$.

Finally, the third central moment of the Poisson distribution with parameter $\lambda$ is $\mathbb E(\bar X_1^3)=\lambda$ hence $$ \mathrm{Cov}(M,S^2)=\frac\lambda{n}. $$

$\endgroup$
  • 1
    $\begingroup$ Thanks for your answer. I haven't fully understood it, and it will take me some time to analyze what you wrote, yet I have a question. I had an intuition that the covariance should get close to zero (because it becomes almost like Normal distribution), when \lambda becomes very large. From your answer, it looks like it grows when \lambda grows. Is my intuition wrong? Thanks again $\endgroup$ – Andrey Rubshtein Nov 12 '12 at 13:52
  • 2
    $\begingroup$ The argument needs some refining. When $\lambda\to\infty$, what happens is really that the scaled and centered versions are more and more gaussian-like, hence symmetric. The proper scaling is to center each $X_k$ and to divide it by $\sqrt{\lambda}$ hence one should divide $M$ by $\sqrt{\lambda}$ and $S^2$ by $\lambda$. Thus, $\mathrm{Cov}(M,S^2)/\lambda^{3/2}$ should go to zero when $\lambda\to\infty$ (and it does). $\endgroup$ – Did Nov 12 '12 at 13:59
  • $\begingroup$ Hi Did, It's a very late reply, not sure whether you will see it or not. Intuitively, I can see that after centering the variables, we did not change $S^2$, and we did not change the $Cov(\bar{X},S^2)$. But I am not sure how to show that the covariance doesn't change by centering the variable. $\endgroup$ – Nan Sep 25 '17 at 20:46
  • 1
    $\begingroup$ @Nan For every constants $x$ and $y$, $\mathrm{Cov}(X-x,Y-y)=\mathrm{Cov}(X,Y)$. Is this your question? $\endgroup$ – Did Sep 25 '17 at 20:58
  • $\begingroup$ @Did Yes! Thank you so much! $\endgroup$ – Nan Sep 25 '17 at 21:53
3
$\begingroup$

One can solve this more generally. For any random variable whose moments exist, the covariance between the sample mean $M$ and the sample variance $S^2$ is:

$$\text{Cov}(M, S^2) = \frac{\mu_3}{n}$$

where $\mu_3$ denotes the $3^{\text{rd}}$ central moment of the parent random variable $X$.

If $X \sim \text{Poisson}(\lambda)$, then $\mu_3 = \lambda$, so the answer is $\large\frac{\lambda}{n}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.