# Distribution of sample variance of Bernoulli variables

I am facing the following problem, given $$X_1 ... X_n$$ a random sample of $$Bernoulli(\theta)$$ variables find the distribution of the sample variance $$S^2 = \frac{1}{n} \sum_i(\bar{X} - X_i)^2$$.

I have demonstrated that $$S^2 = \bar{X} (1 - \bar{X})$$ and i know $$\bar{nX}$$ has distribution $$Binomial (n, \theta)$$ but I have not been able to deduce the distribution of $$S^2$$.

• " I have demonstrated that $S^2 = \bar{X} (1 - \bar{X})$ ": Are you sure that this true? Mar 14, 2020 at 19:52
• @callculus Isn't it? math.stackexchange.com/questions/3546180/… agrees on the equality Mar 14, 2020 at 20:07
• It seems so, but I haven´t checked it. But I think the distribution of $S^2$ is so unhandsome that usually the central limit theorem is used. Mar 14, 2020 at 20:08
• You can get a proper bar across a wider expression, e.g. in $\overline{nX}$, using \overline instead of \bar. Mar 14, 2020 at 20:51

You know the distribution of $$\overline X$$ and you’ve correctly determined that $$S^2$$ is a function of $$\overline X$$. Then the distribution of $$S^2$$ is just the composition of the two: The probability of a value of $$S^2$$ is the sum of the probabilities of the values of $$\overline X$$ that are its preimages; that is, for all integers $$k$$ from $$0$$ to $$\left\lfloor\frac n2\right\rfloor$$ we have:

$$P\left(S^2=\frac kn\left(1-\frac kn\right)\right)=P\left(\overline X=\frac kn\lor\overline X=1-\frac kn\right)= \begin{cases} \binom nk\theta^k(1-\theta)^k&k=\frac n2\;,\\ \binom nk\left(\theta^k(1-\theta)^{n-k}+\theta^{n-k}(1-\theta)^k\right)&\text{otherwise}\;.\\ \end{cases}$$

• "otherwise" seems to raise a variety of issues, as does having an answer in terms of $k$ Mar 15, 2020 at 1:04
• @Henry: I'd forgotten to normalize $k$; all the $k$s should have been $\frac kn$ in the upper line. I've also made the range of $k$ explicit now. Is it OK now? Or is there a more fundamental issue? I'm not sure what you meant by issues being raised by "otherwise" and by the answer being in terms of $k$. Mar 15, 2020 at 2:43
• The point I was aiming at is that there are a lot of "otherwise" values of $c$ for which $P(S^2=c) =0$. There are also cases where two different values of $k$ correspond to the same $S^2$. So you could instead have something like $P(S^2=c)$ being $\binom n{n/2}\left(\theta (1-\theta)\right)^{n/2}$ when $c=\frac14$ and $n$ is an even integer; being $\binom n{n\left(1-\sqrt{1-4c^2}\right)/2}\left(\theta (1-\theta)\right)^{n(1-\sqrt{1-4c^2})/2}\left(\theta^{n\sqrt{1-4c^2}} +(1-\theta)^{n\sqrt{1-4c^2}}\right)$ when $n-n\sqrt{1-4c}$ is a non-negative even integer $<n$; and being $0$ otherwise. Mar 15, 2020 at 11:31
• @Henry: I think the cases where two different values of $k$ correspond to the same $S^2$ are correctly covered in the answer. You're right that "otherwise" was imprecise; I was relying on it being clear what values of $k$ it makes sense to substitute. I made that explicit in response to your first comment. I think it should be OK now? Mar 15, 2020 at 11:35
• Well, I'm still not saying that it's $0$ otherwise. Perhaps we just have different ambitions for rigour and explicitness :-) Mar 15, 2020 at 11:36

Noting that $$\overline{X} \approx \mathcal{N}(\theta,\frac{\theta(1-\theta)}{n})$$ by the Central Limit Theorem, if we apply the second-order delta method to $$g(\theta) = \theta(1-\theta)$$, it follows that we can get a good continuous approximation to the PMF $$f_{S^{2}}$$ of $$S^{2}$$ via

$$f_{S^{2}}(y) \approx \frac{|C|}{\sigma\sqrt{2\pi}}\sqrt{\frac{n}{ (\frac{B}{C})^{2} + 4(Cy - \frac{A}{C}) }}\left( e^{\frac{-n}{8\sigma^{2}}(\sqrt{(\frac{B}{C})^{2} + 4(Cy - \frac{A}{C}) } - \frac{B}{C})^{2} } + e^{\frac{-n}{8\sigma^{2}}(\sqrt{(\frac{B}{C})^{2} + 4(Cy - \frac{A}{C}) } + \frac{B}{C})^{2} } \right) 1_{0 < y < 1/4}$$

where here $$A = \overline{X}(1-\overline{X})$$, $$B = 1 - 2\overline{X}$$, $$C = -1$$ and $$\sigma^{2} = \overline{X}(1-\overline{X})$$.

Though this is not the exact distribution of $$S^{2}$$ as given in Joriki's answer, you'll find this to be a very good continuous approximation to the sampling distribution of $$S^{2}$$, even when $$n$$ is quite small.