2
$\begingroup$

I am facing the following problem, given $X_1 ... X_n$ a random sample of $Bernoulli(\theta)$ variables find the distribution of the sample variance $S^2 = \frac{1}{n} \sum_i(\bar{X} - X_i)^2$.

I have demonstrated that $S^2 = \bar{X} (1 - \bar{X})$ and i know $\bar{nX}$ has distribution $Binomial (n, \theta)$ but I have not been able to deduce the distribution of $S^2$.

$\endgroup$
4
  • 1
    $\begingroup$ " I have demonstrated that $S^2 = \bar{X} (1 - \bar{X})$ ": Are you sure that this true? $\endgroup$ Mar 14, 2020 at 19:52
  • $\begingroup$ @callculus Isn't it? math.stackexchange.com/questions/3546180/… agrees on the equality $\endgroup$
    – Zanzag
    Mar 14, 2020 at 20:07
  • 1
    $\begingroup$ It seems so, but I haven´t checked it. But I think the distribution of $S^2$ is so unhandsome that usually the central limit theorem is used. $\endgroup$ Mar 14, 2020 at 20:08
  • $\begingroup$ You can get a proper bar across a wider expression, e.g. in $\overline{nX}$, using \overline instead of \bar. $\endgroup$
    – joriki
    Mar 14, 2020 at 20:51

2 Answers 2

0
$\begingroup$

You know the distribution of $\overline X$ and you’ve correctly determined that $S^2$ is a function of $\overline X$. Then the distribution of $S^2$ is just the composition of the two: The probability of a value of $S^2$ is the sum of the probabilities of the values of $\overline X$ that are its preimages; that is, for all integers $k$ from $0$ to $\left\lfloor\frac n2\right\rfloor$ we have:

$$ P\left(S^2=\frac kn\left(1-\frac kn\right)\right)=P\left(\overline X=\frac kn\lor\overline X=1-\frac kn\right)= \begin{cases} \binom nk\theta^k(1-\theta)^k&k=\frac n2\;,\\ \binom nk\left(\theta^k(1-\theta)^{n-k}+\theta^{n-k}(1-\theta)^k\right)&\text{otherwise}\;.\\ \end{cases} $$

$\endgroup$
5
  • $\begingroup$ "otherwise" seems to raise a variety of issues, as does having an answer in terms of $k$ $\endgroup$
    – Henry
    Mar 15, 2020 at 1:04
  • $\begingroup$ @Henry: I'd forgotten to normalize $k$; all the $k$s should have been $\frac kn$ in the upper line. I've also made the range of $k$ explicit now. Is it OK now? Or is there a more fundamental issue? I'm not sure what you meant by issues being raised by "otherwise" and by the answer being in terms of $k$. $\endgroup$
    – joriki
    Mar 15, 2020 at 2:43
  • $\begingroup$ The point I was aiming at is that there are a lot of "otherwise" values of $c$ for which $P(S^2=c) =0$. There are also cases where two different values of $k$ correspond to the same $S^2$. So you could instead have something like $P(S^2=c)$ being $\binom n{n/2}\left(\theta (1-\theta)\right)^{n/2}$ when $c=\frac14$ and $n$ is an even integer; being $\binom n{n\left(1-\sqrt{1-4c^2}\right)/2}\left(\theta (1-\theta)\right)^{n(1-\sqrt{1-4c^2})/2}\left(\theta^{n\sqrt{1-4c^2}} +(1-\theta)^{n\sqrt{1-4c^2}}\right)$ when $n-n\sqrt{1-4c}$ is a non-negative even integer $<n$; and being $0$ otherwise. $\endgroup$
    – Henry
    Mar 15, 2020 at 11:31
  • $\begingroup$ @Henry: I think the cases where two different values of $k$ correspond to the same $S^2$ are correctly covered in the answer. You're right that "otherwise" was imprecise; I was relying on it being clear what values of $k$ it makes sense to substitute. I made that explicit in response to your first comment. I think it should be OK now? $\endgroup$
    – joriki
    Mar 15, 2020 at 11:35
  • $\begingroup$ Well, I'm still not saying that it's $0$ otherwise. Perhaps we just have different ambitions for rigour and explicitness :-) $\endgroup$
    – joriki
    Mar 15, 2020 at 11:36
0
$\begingroup$

Noting that $\overline{X} \approx \mathcal{N}(\theta,\frac{\theta(1-\theta)}{n})$ by the Central Limit Theorem, if we apply the second-order delta method to $g(\theta) = \theta(1-\theta)$, it follows that we can get a good continuous approximation to the PMF $f_{S^{2}}$ of $S^{2}$ via

$ f_{S^{2}}(y) \approx \frac{|C|}{\sigma\sqrt{2\pi}}\sqrt{\frac{n}{ (\frac{B}{C})^{2} + 4(Cy - \frac{A}{C}) }}\left( e^{\frac{-n}{8\sigma^{2}}(\sqrt{(\frac{B}{C})^{2} + 4(Cy - \frac{A}{C}) } - \frac{B}{C})^{2} } + e^{\frac{-n}{8\sigma^{2}}(\sqrt{(\frac{B}{C})^{2} + 4(Cy - \frac{A}{C}) } + \frac{B}{C})^{2} } \right) 1_{0 < y < 1/4} $

where here $A = \overline{X}(1-\overline{X})$, $B = 1 - 2\overline{X}$, $C = -1$ and $\sigma^{2} = \overline{X}(1-\overline{X})$.

Though this is not the exact distribution of $S^{2}$ as given in Joriki's answer, you'll find this to be a very good continuous approximation to the sampling distribution of $S^{2}$, even when $n$ is quite small.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.