# Normal Approximation for Maximum Likelihood Estimator

Given $\hat \theta=$ the maximum likelihood estimator for a parameter $\theta$ of a distribution, we know that $$\sqrt{n}(\hat \theta-\theta)\rightarrow^d N(0,V(\hat\theta))$$ where the $\rightarrow^d$ represents convergence in distribution. However, does this imply that: $$\hat\theta\sim_a N\left( \theta,\frac{V(\hat\theta)}{n} \right)?$$

Meaning that, for large enough $n$, the distribution of $\hat\theta$ can be approximated by a normal distribution with the specified mean and variance? $\sim_a$ is just the notation I used to represent "approximately follows". I know that the actual distribution of $\hat\theta$ may not be normal.

• It's hard to answer without a more precise statement of what you're asking. In my reading, the phrase 'for large enough $n$, the distribution of $\hat \theta$ can be approximated by a normal' is a good approximation of what the convergence in distribution result 'means'. – spaceisdarkgreen Feb 17 '18 at 3:33
• Right, but are the specified variance and mean terms correct for that approximation? – jippyjoe4 Feb 17 '18 at 3:36
• Given that the expression $V(\hat \theta)$ is an 'asymptotic scaled variance' defined by your headline result (i.e. don't mistake it for the variance of $\hat \theta$), what you wrote for the approximate distribution of $\hat \theta$ is right. – spaceisdarkgreen Feb 17 '18 at 3:46

Don't forget that (under the "regularity" conditions; inter alia,, well defined and finite Fisher information) you have $$\sqrt{n}( \hat{\theta}_n - \theta )\xrightarrow{D}N(0, \mathcal{I}^{-1}(\theta)),$$
where $\mathcal{I}^{-1}(\theta)$ is the inverse of Fisher information matrix/scalar. Namely, asymptotically, the variance of an MLE reaches the Cramer-Rao lower bound for unbiased estimators (even if the MLE is biased for any finite $n$). Hence, for large enough $n$ indeed $$\hat{\theta}_n \sim_{approx} N(\theta,n^{-1}I^{-1}(\theta)).$$
Whether this approximation is good or bad is the same question as when we can use the CLT (stressed: LIMIT) to say that $$\bar{X}_n \sim N(\mu,n^{-1}\sigma^2),$$ regardless of the original distribution? And the answer is; for symmetric, "well behaved" (i.e., small variance, unimodal, etc.) for fair $n$ it'll be OK, for not so nice distributions - it's not. General answer? Without any further information except for the validity of the regularity conditions, you can only be sure for $n\to \infty$.