1
$\begingroup$

Let $X_{1}, X_{2}, \ldots, X_{n}$ be a sequence of independent, standard Normal, real-valued random variables, and consider the empirical mean $\hat{S}_{n}=\frac{1}{n} \sum_{i=1}^{n} X_{i}$. Since $\hat{S}_{n}$ is again a Normal random variable with zero mean and variance $1 / n$, it follows that for any $\delta>0$, $$ P\left(\left|\hat{S}_{n}\right| \geq \delta\right) \underset{n \rightarrow \infty}{\longrightarrow} 0, $$ and for any interval $A$ by CLT: $$ P\left(\sqrt{n} \hat{S}_{n} \in A\right) \underset{n \rightarrow \infty}{\longrightarrow} \frac{1}{\sqrt{2 \pi}} \int_{A} e^{-x^{2} / 2} d x $$ Note now that by a change of variable $$ P\left(\left|\hat{S}_{n}\right| \geq \delta\right)=1-\frac{1}{\sqrt{2 \pi}} \int_{-\delta \sqrt{n}}^{\delta \sqrt{n}} e^{-x^{2} / 2} d x= \frac{1}{\sqrt{2 \pi}} \int_{|x|> \delta \sqrt n} e^{-x^{2} / 2} d x $$ Now in [Large deviation techniques and applications, Amir Dembo Ofer Zeitouni] it is claimed that $$ \frac{1}{n} \log P\left(\left|\hat{S}_{n}\right| \geq \delta\right) \underset{n \rightarrow \infty}{\longrightarrow}-\frac{\delta^{2}}{2} $$ How do you see it?

$\endgroup$
1
  • $\begingroup$ I think this is related to $\log(2\Phi(-x)) \approx -x^2/2$ for large $x$ $\endgroup$
    – Henry
    Mar 18, 2022 at 15:52

2 Answers 2

2
$\begingroup$

Attempting to flesh out Henry's comment:

The left-hand side is $\frac{1}{n} \log [2(1-\Phi(\delta \sqrt{n}))]$, which has the same limit as $\frac{1}{n} \log(1-\Phi(\delta \sqrt{n}))$ (if the limits exist).

From integration by parts, we have the following Mills ratio bounds for $z>0$: $$\frac{1}{z} - \frac{1}{z^3} < \frac{1-\Phi(z)}{\phi(z)} < \frac{1}{z} - \frac{1}{z^3} + \frac{3}{z^5}.$$ [Perhaps this inequality is overkill for your particular question, is there a simpler result?]

This means $$ -\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) + \frac{1}{n}\log\left(\frac{1}{z} - \frac{1}{z^3}\right) < \frac{1}{n}\log(1-\Phi(z)) < -\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) + \frac{1}{n} \log\left(\frac{1}{z} - \frac{1}{z^3} + \frac{3}{z^5}\right).$$ That is, $$ \frac{1}{n} \log(1-z^{-2}) <\frac{1}{n} \log(1-\Phi(z)) - \left(-\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) - \frac{\log z}{n}\right) < \frac{1}{n} \log(1-z^{-2} + 3 z^{-4})$$

If you plug in $z=\delta\sqrt{n}$ you can show $-\frac{z^2}{2n} + \frac{1}{2n} \log(2\pi) - \frac{\log z}{n} \to -\frac{\delta^2}{2}$ and that the two outer bounds converge to zero.

$\endgroup$
1
$\begingroup$

While the existing answer is nice and absolutely correct, it still misses important general techniques from the large deviation theory, so I will add mine.

Concerning the upper bound, it is usually obtained with the help of Chernoff inequality: $$ \mathrm P(\hat S_n\ge \delta) \le \frac{\mathrm E[e^{\lambda \hat S_n}]}{e^{\lambda \delta}} = \exp\Big\{\frac{\lambda^2}{2n} - \lambda \delta\Big\}, \lambda >0, $$ whence, choosing $\lambda = n\delta$ (which minimizes the left-hand side), $$ \mathrm P(\hat S_n\ge \delta) \le e^{-n\delta^2/2}. $$ Similarly, $\mathrm P(\hat S_n\le -\delta) \le e^{-n\delta^2/2}$, so $$ \limsup_{n\to\infty} \frac{1}{n}\log \mathrm P(|\hat S_n|\ge \delta)\le \limsup_{n\to\infty} \frac{1}{n}\Big( -\frac{n\delta^2}2 +\ln 2\Big) = -\frac{\delta^2}2. $$

To show the lower bound, it is often enough to localize the integrand "near the boundary": $$ \mathrm P(|\hat S_n|\ge \delta) = \frac{1}{\sqrt{2 \pi}} \int_{|x|\ge \delta \sqrt n} e^{-x^{2} / 2} d x\ge \frac{1}{\sqrt{2 \pi}} \int_{\delta \sqrt n}^{\delta\sqrt{n}+1} e^{-x^{2} / 2} d x \ge \frac{1}{\sqrt{2 \pi}}\cdot \exp\Big\{-\frac{(\delta\sqrt{n}+1)^2}{2}\Big\}, $$ whence $$ \liminf_{n\to\infty} \frac{1}{n}\log \mathrm P(|\hat S_n|\ge \delta)\ge \liminf_{n\to\infty} \frac{1}{n} \Big(-\frac{(\delta\sqrt{n}+1)^2}{2}-\ln \sqrt{2\pi}\Big) = -\frac{\delta^2}{2}. $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .