# MLE for normal distribution with mean and variance unknown, consistency and histograms

Consider $$X_1, X_2, \ldots, X_n$$ i.i.d $$N(\mu, \sigma^2),$$ where both parameters are unknown, and consider estimation of $$\sigma^2.$$ Consider the MLE for $$\sigma^2.$$ We know it is, $$\hat{\sigma^2_n} = \frac{1}{n} \sum_{i=1}^n \left(X_i-\bar{X} \right)^2.$$

I have shown $$\hat{\sigma^2_n}$$ is biased but consistent for $$\sigma^2.$$ Now, I want to add a concrete example to visualize the property of consistency in $$\hat{\sigma^2_n}$$ and ran into the following hurdle.

I considered $$X_1, X_2, \ldots, X_n$$ i.i.d $$N(2, 3^2)$$. Took random samples of size $$n$$. Have a plot that shows how as $$n$$ increases, $$\hat{\sigma}^2_n \to 9$$ in probability. This makes sense since $$\hat{\sigma^2_n}$$ is consistent.

But then I did the following:

I repeatedly ($$100,000$$ times) took samples of size $$n=20$$, calculated $$\hat{\sigma}^2_n$$, and visualized with histogram to get:

I repeatedly ($$100,000$$ times) took samples of size $$n=10,000$$, calculated $$\hat{\sigma}^2_n$$, and visualized with histogram to get:

My question is, are these histograms showing that $$\hat{\sigma^2_n}$$ is asymptotically unbiased (not always true for all consistent estimators but true for this one)? Or is it showing that $$\hat{\sigma^2_n}$$ is also an efficient estimator (what is the Cramer Rao Lower Bound when $$\mu$$ is also unknown, and is the asymptotic variance of $$\hat{\sigma^2_n}$$ equal to it)?

Your histograms seem to suggest that for very large $$n$$ the distribution concentrates around $$9$$ which is $$\sigma^2$$.
You might add a vertical line to each of the means of the variance estimates, with the first somewhere presumably near $$9.47$$ and the second much nearer $$9$$ to suggest asymptotically unbiasness. If this is difficult to visualise (the first line will be closer to $$10$$ than the second) then repeat the first with perhaps $$n=10$$ and a vertical line near $$10$$
• Your first histogram does not in itself show bias visually, which is why I suggested adding a vertical line. As for efficiency, this is a relative concept and will not be shown by histograms: you might instead take sever different $n$ and show the simulated mean-square error for each compared with the CR lower bound for each $n$ (though since the MLE is biased, you might get unexpected results) May 3, 2020 at 14:16