6
$\begingroup$

Suppose that $ (\Omega,\Sigma,\mathsf{P}) $ is a probability space and that $ (X_{k})_{k \in \mathbb{N}} $ is a sequence of i.i.d. Bernoulli trials on $ (\Omega,\Sigma,\mathsf{P}) $, each with probability of success $ p \in (0,1) $. If we define another sequence $ (\hat{P}_{n})_{n \in \mathbb{N}} $ of random variables on $ (\Omega,\Sigma,\mathsf{P}) $ by $$ \forall n \in \mathbb{N}: \qquad \hat{P}_{n} \stackrel{\text{df}}{=} \frac{1}{n} \sum_{k = 1}^{n} X_{k}, $$ then according to the Central Limit Theorem, we have $$ \forall z \in \mathbb{R}: \qquad \lim_{n \to \infty} \mathsf{P} \! \left( \frac{\hat{P}_{n} - p}{\sqrt{p (1 - p) / n}} \leq z \right) = \Phi(z), $$ where $ \Phi $ denotes the standard normal c.d.f. For each $ n \in \mathbb{N} $, we call $ \hat{P}_{n} $ a sample proportion for a sample of size $ n $.

When most statistics textbooks discuss confidence intervals for a sample proportion, they implicitly claim that $$ \frac{\hat{P}_{n} - p}{\sqrt{\hat{P}_{n} (1 - \hat{P}_{n}) / n}} \stackrel{\text{d}}{\longrightarrow} \operatorname{N}(0,1), $$ which is the same as saying that $$ \forall z \in \mathbb{R}: \qquad \lim_{n \to \infty} \mathsf{P} \! \left( \frac{\hat{P}_{n} - p}{\sqrt{\hat{P}_{n} (1 - \hat{P}_{n}) / n}} \leq z \right) = \Phi(z). $$ However, I was unable to rigorously establish this claim using the Central Limit Theorem.

Could anyone kindly provide references? Thanks!

$\endgroup$
2
  • $\begingroup$ It is not pure CLT, since the variance is not truly known. Think about the hypothesis test angle: in this case you postulate a value for $p $ in the null hypothesis and ask whether the probability of seeing data like yours is significant under that assumption. Then this calculation is done under this assumption, so that the variance is "known", under the assumption. $\endgroup$
    – Ian
    Jul 15, 2016 at 0:30
  • $\begingroup$ A CI is like saying which null hypotheses you would reject and which you would retain, given the data you found and the level of significance you chose. $\endgroup$
    – Ian
    Jul 15, 2016 at 0:33

1 Answer 1

Reset to default
5
$\begingroup$

The most straightforward proof of this result requires knowledge of Slutsky's theorem, which in turn requires the concept of convergence in probability. Write $$\frac{\hat{P}_{n} - p}{\sqrt{\hat{P}_{n} (1 - \hat{P}_{n}) / n}}= \frac{\hat{P}_{n} - p}{\sqrt{p(1-p) / n}} \cdot \sqrt{ \frac{p(1-p)}{\hat P_n(1-\hat P_n)}}, $$ a product of two factors. The first factor converges in distribution to the standard normal, by the central limit theorem. The second factor converges almost surely to the constant value $1$, by the law of large numbers. Now apply Slutsky's theorem, since convergence a.s. implies convergence in probability.

$\endgroup$
7
  • 1
    $\begingroup$ Note that you could've used WLLN directly, rather than using SLLN to infer WLLN. $\endgroup$
    – Ian
    Jul 15, 2016 at 0:59
  • 1
    $\begingroup$ @Ian Yep, the WLLN gets you $\hat P_n\to p$ in probability. To show the square rooted thing converges in prob to $1$, you apply the fact that $X_n\to c$ in probability implies $h(X_n)\to h(c)$ in probability if $h$ is continuous at $c$. $\endgroup$
    – grand_chat
    Jul 15, 2016 at 1:11
  • $\begingroup$ @grand_chat: Thank you for this solution. It has been a while since I last took a class in mathematical statistics, so I had totally forgotten about Slutsky’s Theorem. By the way, do you happen to know of any references that discuss how quickly $ \dfrac{\hat{P}_{n} - p}{\sqrt{\hat{P}_{n} (1 - \hat{P}_{n}) / n}} $ converges in distribution to $ \operatorname{N}(0,1) $? $\endgroup$
    – Transcendental
    Jul 15, 2016 at 15:11
  • 1
    $\begingroup$ @Transcendental The difference between the $N(0,1)$ distribution and the distribution of the first factor first factor can be estimated using the Berry-Esseen theorem (though the bound you get might be far from optimal). The difference between the second factor and $1$ in probability can be estimated using Chebyshev's inequality. To combine the two you'd want to look back over the proof of Slutsky's theorem, I guess. $\endgroup$
    – Ian
    Jul 15, 2016 at 16:15
  • 1
    $\begingroup$ @Transcendental Incidentally, on this subject of r.v.s with exactly a second moment and no higher, one of them is an r.v. which is drawn from a distribution with pdf $f_n(x)=1_{[1,\infty)}(x) x^{-3-a_n}$ with probability $\frac{b_n}{\sum_{n=1}^\infty b_n}$, where $a_n>0$, $a_n \to 0$, $b_n>0$, and $b_n$ goes to zero sufficiently fast. Such a r.v. has exactly a second moment and no higher moments. It is an interesting question to look into the convergence rate of the CLT in this case. It's a fun numerical project. (Of course you cannot use an infinite table, but merely a finite truncation.) $\endgroup$
    – Ian
    Jul 30, 2016 at 19:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .