I have a sequence of independent, but non-identically distributed Bernoulli random variables $X_i$'s taking on value $1$ with probability $p_{i1}\cdot p_{i2}$, where $i=1,\ldots,n$. Let $X=\sum_{i=1}^n X_i$. Using Lyapunov central limit theorem, I have approximated the probability $\Pr[X \geq k]$ by standard normal distribution. However, for some values of $k$ the approximated value is not close to the exact value. I wanted to ask that why this happens? Thanks in advance for your help.

  • 1
    $\begingroup$ It would definitely be helpful to give an explicit example $\endgroup$ – spaceisdarkgreen Sep 5 '18 at 3:58
  • 1
    $\begingroup$ To clarify: Is it the case that the approximation doesn't hold for small values of $n$? Or that the approximation doesn't seem to get better as $n$ gets larger? (Thanks) $\endgroup$ – Patrick Hew Sep 5 '18 at 7:43
  • $\begingroup$ The normal approximation may give poor results in the tail, in the sense of relative errors, where tail will depend on the particular question $\endgroup$ – Henry Sep 5 '18 at 8:01
  • $\begingroup$ For instance, let $n=17618$. The approximated value by central limit theorem for small values of $k$ is good but as k increases the error between approximated value and exact value increases. $\endgroup$ – Carol Sep 5 '18 at 18:32
  • $\begingroup$ @PatrickHew The approximation doesn't hold as $k$ increases. $\endgroup$ – Carol Sep 6 '18 at 16:51


Notation: Let $F_n$ be the cumulative distribution function for $X = \sum_{I=1}^n X_i$, and let $F$ be the cumulative distribution function $Z$ where $Z$ follows a standard normal distribution.

The result that you have only establishes that for any fixed $k$, $P[Z \geq k]$ will be an increasingly good approximation to $P[X \geq k]$ as $n \rightarrow \infty$.


You have convergence in distribution (by the Lyapunov central limit theorem): for any $x \in \mathbb{R}$, $F_n(x) \rightarrow F(x)$ as $n \rightarrow \infty$. (This holds for any real value $x$ since $F$ is continuous on the real numbers $\mathbb{R}$.)

But (I think that) what you're seeing is that for any given $n$, the closeness of $F_n(x)$ to $F(x)$ can vary with $x$. Hence in particular, the closeness of $F_n(k)$ to $F(k)$ can vary as $k$ increases. Therefore since $P[X \geq k] = 1 - F_n(k)$ and $P[Z \geq k] = 1 - F(k)$, the closeness of $P[Z \geq k]$ to $P[X \geq k]$ can also vary as $k$ increases.

In other words, the convergence in distribution is pointwise convergence and not uniform convergence. That is, in establishing that for any $x \in \mathbb{R}$, $F_n(x) \rightarrow F(x)$ as $n \rightarrow \infty$, the following is true: $$ \text{pointwise convergence:}\ \text{for any}\ x \in \mathbb{R}\ \text{and for any}\ \epsilon > 0, \text{there exists}\ N > 0\ \text{such that if}\ n \geq N \ \text{then}\ |F_n(x) - F(x) |< \epsilon $$ The following is not necessarily true: $$ \text{uniform convergence:}\ \text{for any}\ \epsilon > 0, \text{there exists}\ N > 0\ \text{such that for any}\ x \in \mathbb{R}, \text{if}\ n \geq N \ \text{then}\ |F_n(x) - F(x) |< \epsilon $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.