# what is the intuition behind Delta method?

I'm trying to learn the delta method in probability but couldn't quite get the hang of it. For example: trying to solve a problem from the book statistical Inference :

Consider a random sample from $\mathrm{Beta}(\alpha ,\beta)$ distribution $\alpha > 0, \beta > 0$, use delta method to find approximate distribution of $T_n = {\bar X_n}(1- {\bar X_n})$ and find the approximate distribution of ${\bar X_n}$ when $n$ is large.

My try:

a. By delta method,

$$\sqrt{n} [f(x) - f(\mu)] \stackrel{\mathcal{D}}{\longrightarrow} N(0,\sigma^2[f'(x)]^2)$$

Here, $f(x) = x(1-x)$ and $f(\mu) = \mu(1-\mu)$

$\implies \sqrt{n} [x(1-x) - \mu(1-\mu)] \stackrel{\mathcal{D}}{\longrightarrow} N(0,[1-2\mu]^2)$

$\implies \sqrt{n} [\log(x(1-x)) - \log(\mu(1-\mu))] \stackrel{\mathcal{D}}{\longrightarrow} N(0,2 \log (1-2\mu))$ $\implies \sqrt{n} [\log(x(1-x)) - \log(\mu(1-\mu))] \stackrel{\mathcal{D}}{\longrightarrow} N(0,\mu))$

b. For the second one I think we should just use the central limit theorem to say the mean follows normal distribution with mean = 0 but what is the variance? Is it 1?

-
You're very close to getting it. First, just plug $\mu$ directly into the formula for $f(x) = x(1-x)$. Think about what the function applied to the mean value is (it's not $k\theta$). Secondly, you are correct to think of the CLT for the approximate distribution of $\bar{X}_{n}$.. but why would it have mean $0$? It won't converge to a standard normal unless you subtract the mean and divide by the standard deviation... so what if you don't do these operations? – Mr. F Mar 19 '12 at 18:45
@EMS I don't understand "what function applied to mean value is?" Is it $k \theta$ because it is the expected value for beta ditribution and in the second part would it have the mean and the variance of beta instead of standard normal? – Sunil Mar 19 '12 at 18:49
$f(x) = x(1-x)$, so $f(\mu) = \mu(1-\mu)$. Just follow the exact procedure as in this example, but substituting in the properties of your distribution and your function. – Mr. F Mar 19 '12 at 19:01
@EMS Does the above seem correct? – Sunil Mar 19 '12 at 19:18

First, let $X_{i}$ be distributed as a Beta distribution with parameters $\alpha$ and $\beta$. Then it has mean $\mu = \frac{\alpha}{(\alpha+\beta)}$ and variance $\sigma^{2} = \frac{\alpha\beta}{(\alpha+\beta)^{2}(\alpha+\beta+1)}$.

Then $\bar{X}_{n} = \frac{1}{n}\sum_{i=1}^{n}X_{i}$, and this will have the same mean as the common mean of all of the $X_{i}$.

The central limit theorem tells us that $$\sqrt{n}\biggl[ \bar{X}_{n} - \mu\biggr] \to \mathcal{N}(0,\sigma^{2}).$$

From this knowledge, we can apply the Delta method with the statistic function of interest being $T_{n} = \bar{X}_{n}(1-\bar{X}_{n})$, or more simply $T(x) = x(1-x)$. The Delta method then tells us that $$\sqrt{n}\biggl[ T(\bar{X}_{n}) - T(\mu)\biggr] \to \mathcal{N}(0,\sigma^{2}[T'(\mu)]^{2}).$$

Now, $T'(x) = 1-2x$, and so $$[T'(\mu)]^{2} = [1-\frac{2\alpha}{(\alpha + \beta)}]^{2} = 4\biggl[\frac{\alpha}{(\alpha+\beta)}\biggr]^{2} -4\biggl[\frac{\alpha}{(\alpha+\beta)}\biggr] + 1.$$

So, the variance for the distribution of $\sqrt{n}[T-T(\mu)]$ is given by: $$\sigma_{T}^{2} = \biggl(\frac{\alpha\beta}{(\alpha+\beta)^{2}(\alpha+\beta+1)}\biggr)\biggl(4\biggl[\frac{\alpha}{(\alpha+\beta)}\biggr]^{2} -4\biggl[\frac{\alpha}{(\alpha+\beta)}\biggr] + 1\biggr).$$

Hopefully you can take it from there. All that remains is adjusting the given distribution of $\sqrt{n}[T-T(\mu)]$ to get just the distribution of $T$, and this should be discussed anywhere that the CLT is discussed.

As for the 'intuition' behind this method, it is a very similar idea to a transformation of variables. The Wikipedia proof for the univariate case does a good job of showing what happens.

When you expand the function $g(X)$ around the point $\theta$ and you assume only a linear approximation, then what you're left with as a scale factor for the term $(X-\theta)$ is $g'(\theta)$, which just comes from simple Taylor series approximation. Dividing both sides by $g'(\theta)$ leaves you with $X-\theta$ on the right hand side, which is something with a known asymptotic distribution. That means all the stuff on the left hand side has to have that same asymptotic distribution. Multiplying by $g'(\theta)$ then gives the result. This also shows why the assumption that $g'(x)$ is not $0$ at $\theta$ (although this is not strictly necessary if you make higher order arguments).

-