# Limit of some probability of the empirical mean of independent random variables

Let $$X_{1}, X_{2}, \ldots, X_{n}$$ be a sequence of independent, standard Normal, real-valued random variables, and consider the empirical mean $$\hat{S}_{n}=\frac{1}{n} \sum_{i=1}^{n} X_{i}$$. Since $$\hat{S}_{n}$$ is again a Normal random variable with zero mean and variance $$1 / n$$, it follows that for any $$\delta>0$$, $$P\left(\left|\hat{S}_{n}\right| \geq \delta\right) \underset{n \rightarrow \infty}{\longrightarrow} 0,$$ and for any interval $$A$$ by CLT: $$P\left(\sqrt{n} \hat{S}_{n} \in A\right) \underset{n \rightarrow \infty}{\longrightarrow} \frac{1}{\sqrt{2 \pi}} \int_{A} e^{-x^{2} / 2} d x$$ Note now that by a change of variable $$P\left(\left|\hat{S}_{n}\right| \geq \delta\right)=1-\frac{1}{\sqrt{2 \pi}} \int_{-\delta \sqrt{n}}^{\delta \sqrt{n}} e^{-x^{2} / 2} d x= \frac{1}{\sqrt{2 \pi}} \int_{|x|> \delta \sqrt n} e^{-x^{2} / 2} d x$$ Now in [Large deviation techniques and applications, Amir Dembo Ofer Zeitouni] it is claimed that $$\frac{1}{n} \log P\left(\left|\hat{S}_{n}\right| \geq \delta\right) \underset{n \rightarrow \infty}{\longrightarrow}-\frac{\delta^{2}}{2}$$ How do you see it?

• I think this is related to $\log(2\Phi(-x)) \approx -x^2/2$ for large $x$ Mar 18, 2022 at 15:52

Attempting to flesh out Henry's comment:

The left-hand side is $$\frac{1}{n} \log [2(1-\Phi(\delta \sqrt{n}))]$$, which has the same limit as $$\frac{1}{n} \log(1-\Phi(\delta \sqrt{n}))$$ (if the limits exist).

From integration by parts, we have the following Mills ratio bounds for $$z>0$$: $$\frac{1}{z} - \frac{1}{z^3} < \frac{1-\Phi(z)}{\phi(z)} < \frac{1}{z} - \frac{1}{z^3} + \frac{3}{z^5}.$$ [Perhaps this inequality is overkill for your particular question, is there a simpler result?]

This means $$-\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) + \frac{1}{n}\log\left(\frac{1}{z} - \frac{1}{z^3}\right) < \frac{1}{n}\log(1-\Phi(z)) < -\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) + \frac{1}{n} \log\left(\frac{1}{z} - \frac{1}{z^3} + \frac{3}{z^5}\right).$$ That is, $$\frac{1}{n} \log(1-z^{-2}) <\frac{1}{n} \log(1-\Phi(z)) - \left(-\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) - \frac{\log z}{n}\right) < \frac{1}{n} \log(1-z^{-2} + 3 z^{-4})$$

If you plug in $$z=\delta\sqrt{n}$$ you can show $$-\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) - \frac{\log z}{n} \to -\frac{\delta^2}{2}$$ and that the two outer bounds converge to zero.

While the existing answer is nice and absolutely correct, it still misses important general techniques from the large deviation theory, so I will add mine.

Concerning the upper bound, it is usually obtained with the help of Chernoff inequality: $$\mathrm P(\hat S_n\ge \delta) \le \frac{\mathrm E[e^{\lambda \hat S_n}]}{e^{\lambda \delta}} = \exp\Big\{\frac{\lambda^2}{2n} - \lambda \delta\Big\}, \lambda >0,$$ whence, choosing $$\lambda = n\delta$$ (which minimizes the left-hand side), $$\mathrm P(\hat S_n\ge \delta) \le e^{-n\delta^2/2}.$$ Similarly, $$\mathrm P(\hat S_n\le -\delta) \le e^{-n\delta^2/2}$$, so $$\limsup_{n\to\infty} \frac{1}{n}\log \mathrm P(|\hat S_n|\ge \delta)\le \limsup_{n\to\infty} \frac{1}{n}\Big( -\frac{n\delta^2}2 +\ln 2\Big) = -\frac{\delta^2}2.$$

To show the lower bound, it is often enough to localize the integrand "near the boundary": $$\mathrm P(|\hat S_n|\ge \delta) = \frac{1}{\sqrt{2 \pi}} \int_{|x|\ge \delta \sqrt n} e^{-x^{2} / 2} d x\ge \frac{1}{\sqrt{2 \pi}} \int_{\delta \sqrt n}^{\delta\sqrt{n}+1} e^{-x^{2} / 2} d x \ge \frac{1}{\sqrt{2 \pi}}\cdot \exp\Big\{-\frac{(\delta\sqrt{n}+1)^2}{2}\Big\},$$ whence $$\liminf_{n\to\infty} \frac{1}{n}\log \mathrm P(|\hat S_n|\ge \delta)\ge \liminf_{n\to\infty} \frac{1}{n} \Big(-\frac{(\delta\sqrt{n}+1)^2}{2}-\ln \sqrt{2\pi}\Big) = -\frac{\delta^2}{2}.$$