7
$\begingroup$

Suppose that I have a Poisson distribution $P(\lambda)$.

Let $X_1,X_2,\ldots,X_n$ be independent random variables from the distribution mentioned above.

Let us define sample variance $S^2 = \frac{1}{n-1} \sum (X_i - M)^2 $ and sample mean as $M = \frac{1}{n}\sum X_i $.

I want to find the covariance, $Cov(M,S^2)$. I've seen from this answer that when the distribution is symmetric, they are uncorrelated, which makes it zero. I also know that for large $\lambda$ values Poisson distribution is very close to Gaussian distribution, thus becoming symmetric, and probably the covariance is close to zero.

However, I want to find the exact value of $Cov(M,S^2)$, as I am working with small values of $\lambda$

Currently I tried the following:

$Cov(M,S^2) =E( (M-\lambda) (S^2-\lambda) ) = E(MS^2)-\lambda E(M) - \lambda E(S^2) + \lambda^2$

Thus, $Cov(M,S^2) = E(MS^2) - \lambda^2 - \lambda^2 + \lambda^2 = E(MS^2) -\lambda^2$

I am having trouble with calculating $E(MS^2)$.

Any idea?

$\endgroup$
3
  • $\begingroup$ A square is missing in the definition of $S^2$, isn't it? $\endgroup$
    – Davide Giraudo
    Nov 12, 2012 at 13:00
  • $\begingroup$ @DavideGiraudo, you are right, I made a typo. Thanks for pointing that out. $\endgroup$
    – Andrey Rubshtein
    Nov 12, 2012 at 13:09
  • 1
    $\begingroup$ We can expand the square in the definition of $S^2$ to get a more simple expression, then express $E(MS^2)$ as a double sum. You will have to distinguish when the indexes are the same or not. $\endgroup$
    – Davide Giraudo
    Nov 12, 2012 at 13:13

2 Answers 2

Reset to default
7
$\begingroup$

One can solve this more generally. For any random variable whose moments exist, the covariance between the sample mean $M$ and the sample variance $S^2$ is:

$$\text{Cov}(M, S^2) = \frac{\mu_3}{n}$$

where $\mu_3$ denotes the $3^{\text{rd}}$ central moment of the parent random variable $X$.

If $X \sim \text{Poisson}(\lambda)$, then $\mu_3 = \lambda$, so the answer is $\large\frac{\lambda}{n}$.

$\endgroup$
1
  • 3
    $\begingroup$ Could you either provide a reference to the general result or outline a proof? $\endgroup$
    – W7GVR
    Feb 11, 2020 at 18:20
6
$\begingroup$

Let us first center everything, using $\bar X_k=X_k-\lambda$ and $\bar M=M-\lambda$. Then $$ \mathrm{Cov}(M,S^2)=\mathbb E(\bar MS^2)=\mathbb E(\bar X_1S^2)=\frac1{n-1}\mathbb E(\bar X_1U), $$ where $$ U=\sum\limits_{k=1}^n(\bar X_k-\bar M)^2. $$ Note that $U$ is a linear combination of products $\bar X_k^2$ and $\bar X_k\bar X_i$ for $i\ne k$. Amongst these products, many will not contribute to the expectation of $\bar X_1U$ since $\mathbb E(\bar X_1\bar X_k\bar X_i)=0$ for every $k\ne i$ and $\mathbb E(\bar X_1\bar X_k^2)=0$ for every $k\ne1$.

Hence, one needs only the coefficient of $\bar X_1^2$ in $U$, which is $c_n=\left(\frac{n-1}n\right)^2+(n-1)\frac1{n^2}=\frac{n-1}n$. This yields $\mathbb E(\bar X_1U)=c_n\mathbb E(\bar X_1^3)$ and $\mathrm{Cov}(M,S^2)=\frac1{n-1}c_n\mathbb E(\bar X_1^3)=\frac1n\mathbb E(\bar X_1^3)$.

Finally, the third central moment of the Poisson distribution with parameter $\lambda$ is $\mathbb E(\bar X_1^3)=\lambda$ hence $$ \mathrm{Cov}(M,S^2)=\frac\lambda{n}. $$

$\endgroup$
5
  • 1
    $\begingroup$ Thanks for your answer. I haven't fully understood it, and it will take me some time to analyze what you wrote, yet I have a question. I had an intuition that the covariance should get close to zero (because it becomes almost like Normal distribution), when \lambda becomes very large. From your answer, it looks like it grows when \lambda grows. Is my intuition wrong? Thanks again $\endgroup$
    – Andrey Rubshtein
    Nov 12, 2012 at 13:52
  • 2
    $\begingroup$ The argument needs some refining. When $\lambda\to\infty$, what happens is really that the scaled and centered versions are more and more gaussian-like, hence symmetric. The proper scaling is to center each $X_k$ and to divide it by $\sqrt{\lambda}$ hence one should divide $M$ by $\sqrt{\lambda}$ and $S^2$ by $\lambda$. Thus, $\mathrm{Cov}(M,S^2)/\lambda^{3/2}$ should go to zero when $\lambda\to\infty$ (and it does). $\endgroup$
    – Did
    Nov 12, 2012 at 13:59
  • $\begingroup$ Hi Did, It's a very late reply, not sure whether you will see it or not. Intuitively, I can see that after centering the variables, we did not change $S^2$, and we did not change the $Cov(\bar{X},S^2)$. But I am not sure how to show that the covariance doesn't change by centering the variable. $\endgroup$
    – Nan
    Sep 25, 2017 at 20:46
  • 1
    $\begingroup$ @Nan For every constants $x$ and $y$, $\mathrm{Cov}(X-x,Y-y)=\mathrm{Cov}(X,Y)$. Is this your question? $\endgroup$
    – Did
    Sep 25, 2017 at 20:58
  • $\begingroup$ @Did Yes! Thank you so much! $\endgroup$
    – Nan
    Sep 25, 2017 at 21:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .