# Probability Density Function Converging to Dirac Delta Function Proof

How do I show the following formula is true?

$$\lim_{N\to\infty} p_{\hat{\mu}}(x) = \delta(x-\mu)$$ when $$p_{\hat{\mu}}(x) = \frac{\sqrt{N}}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac12\frac{N}{\sigma^2}(x-\mu)^2\right).$$

It involves the probability density function for the sample mean of independent, identically distributed, Gaussian random variables.

We have: $X_i \sim \mathcal{N}(\mu, \sigma^2)$. $$\hat{\mu} = \frac{1}{N}[x_1 + x_2 + \ldots + x_N]$$ Then, it can be shown that the p.d.f. of $\hat{\mu}$ is: $$p_{\hat{\mu}}(x) = \frac{\sqrt{N}}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac12\frac{N}{\sigma^2}(x-\mu)^2\right).$$ Then, the result is: $$\lim_{N\to\infty} p_{\hat{\mu}}(x) = \delta(x-\mu)$$ Notice that I cannot apply the usual: $$\lim_{x\to\infty}f(x)g(x)$$ since the limit $\lim_{x\to\infty}f(x)$ doesn't exist (a limit which returns infinity is one way in which it does not exist).

If I let $C = \frac{1}{\sqrt{2\pi\sigma^2}}$, $A = -\frac12 \frac{N}{\sigma^2}(x-\mu)^2$, then $$\lim_{N\to\infty}p_{\hat{\mu}}(x) = \frac{C}{2A}\frac{1}{\sqrt{N}}\frac{1}{\exp(AN)}$$

-

This illustrates that $\delta$ is not a function, but a functional: it does not reside in the world of functions but in operators on functions. By definition, you have that $\int_{-\infty}^\infty f(x)\delta(x-\mu)=f(\mu)$ for all nice functions $f$ (continuous and compactly supported), so what you really want to show is $\lim_{n\rightarrow\infty} \int_{-\infty}^\infty p_\hat{\mu}(x)f(x)dx=f(\mu)$ for all compactly supported continuous functions $f$. This is not hard by doing a change of variables that puts $n$ inside $f(x)$ and exploiting continuity.

-
Hello, I am curious how to show $\lim_{N\to\infty}\int_{-\infty}^{\infty}p_{\hat{\mu}}(x)f(x)\,\mathrm{d}x = f(\mu)$. For instance, I actually inserted the expression for the pdf into the integral, but could not see how the $f(x)$ inside the integral becomes the same function $f(\cdot)$ but with a different argument $\mu$. –  jrand Feb 2 '13 at 1:50
$\lim_{N\to\infty} \int_{-\infty}^{\infty}\frac{\sqrt{N}}{\sqrt{2\pi\sigma^2}} \exp(-\frac12 \frac{N}{\sigma^2}(x-2\mu x + \mu^2) f(x)\, \mathrm{d}x$ and I am wondering how to proceed. The goal is to somehow get the integral to result in the same function $f(\mu)$. ** I believe the purpose of doing this is to show that there exists a functional $\lim_{N\to\infty}p_{\hat{\mu}}(x)$ which exhibits the same properties as the delta functional. Then, the limit of the function is the same as the delta functional. –  jrand Feb 2 '13 at 1:53
@jrand: First, shift the integral via $x\rightarrow x-\mu$ to get rid of the $\mu$. Then let $u=\sqrt{N}x$. Then use the dominated convergence theorem. –  Alex R. Feb 2 '13 at 2:03
We wish to show $\int_{-\infty}^{+\infty}f(x)g(x)\,\mathrm{d}x = f(\mu)\Leftrightarrow g(x) = \delta(x-\mu)$. If conditions exist for the dominated convergence theorem to be applicable, $\lim_{N\to +\infty}\int_{-\infty}^{+\infty}f(x)g(x)\,\mathrm{d}x = f(\mu) \Rightarrow \int_{-\infty}^{+\infty}f(x)\lim_{N\to +\infty}g(x)\,\mathrm{d}x = f(\mu) \Leftrightarrow \lim_{N\to +\infty}g(x) = \delta(x - \mu)$. –  jrand Mar 21 '13 at 15:56
$\int_{-\infty}^{+\infty} \sqrt{\frac{N}{2\pi\sigma^2}} \exp\left(-\frac12 \frac{N}{\sigma^2} (x-\mu)^2\right)f(x)\,\mathrm{d}x = \int_{-\infty}^{+\infty}\sqrt{\frac{N}{2\pi\sigma^2}} \exp\left(-\frac12 \frac{N}{\sigma^2}u^2\right)f(u + \mu)\,\mathrm{d}u = \int_{-\infty}^{+\infty}\sqrt{\frac{N}{2\pi\sigma^2}} \exp\left(-\frac{1}{2\sigma^2}x^2\right)f\left(\frac{x}{\sqrt{N}} + \mu\right)\,\frac{\mathrm{d}x}{\sqrt{N}} = \int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi \sigma^2}} \exp\left(-\frac{1}{2\sigma^2}x^2\right)f\left(\frac{x}{\sqrt{N}}+\mu\right)\, \mathrm{d}x$ –  jrand Mar 21 '13 at 15:57

Alternatively, what you want to prove is equivalent to showing that $\hat{\mu}$ converges in probability to $\mu$. That is, for every $\epsilon > 0$,

$$\lim_{n \rightarrow \infty}{P(|\hat{\mu}-\mu|>\epsilon) = 0}$$

This follows from the law of large numbers.

-