1
$\begingroup$

Consider $X_1, X_2, \ldots, X_n$ i.i.d $N(\mu, \sigma^2),$ where both parameters are unknown, and consider estimation of $\sigma^2.$ Consider the MLE for $\sigma^2.$ We know it is, $$\hat{\sigma^2_n} = \frac{1}{n} \sum_{i=1}^n \left(X_i-\bar{X} \right)^2.$$

I have shown $\hat{\sigma^2_n}$ is biased but consistent for $\sigma^2.$ Now, I want to add a concrete example to visualize the property of consistency in $\hat{\sigma^2_n}$ and ran into the following hurdle.

I considered $X_1, X_2, \ldots, X_n$ i.i.d $N(2, 3^2)$. Took random samples of size $n$. Have a plot that shows how as $n$ increases, $\hat{\sigma}^2_n \to 9$ in probability. This makes sense since $\hat{\sigma^2_n}$ is consistent.

But then I did the following:

I repeatedly ($100,000$ times) took samples of size $n=20$, calculated $\hat{\sigma}^2_n$, and visualized with histogram to get:

enter image description here

I repeatedly ($100,000$ times) took samples of size $n=10,000$, calculated $\hat{\sigma}^2_n$, and visualized with histogram to get:

enter image description here

My question is, are these histograms showing that $\hat{\sigma^2_n}$ is asymptotically unbiased (not always true for all consistent estimators but true for this one)? Or is it showing that $\hat{\sigma^2_n}$ is also an efficient estimator (what is the Cramer Rao Lower Bound when $\mu$ is also unknown, and is the asymptotic variance of $\hat{\sigma^2_n}$ equal to it)?

$\endgroup$

1 Answer 1

0
$\begingroup$

Your histograms seem to suggest that for very large $n$ the distribution concentrates around $9$ which is $\sigma^2$.

You might add a vertical line to each of the means of the variance estimates, with the first somewhere presumably near $9.47$ and the second much nearer $9$ to suggest asymptotically unbiasness. If this is difficult to visualise (the first line will be closer to $10$ than the second) then repeat the first with perhaps $n=10$ and a vertical line near $10$

$\endgroup$
3
  • $\begingroup$ That is clear from the histograms. My question is regarding consistency or efficiency of the MLE of the variance (when mean is unknown too) of a random sample of normally distributed random variables. And if either of these properties is reflected in the histograms. You are saying the histograms show asymptotically unbiasedness? But is the MLE efficient? $\endgroup$ May 3, 2020 at 13:36
  • $\begingroup$ Your first histogram does not in itself show bias visually, which is why I suggested adding a vertical line. As for efficiency, this is a relative concept and will not be shown by histograms: you might instead take sever different $n$ and show the simulated mean-square error for each compared with the CR lower bound for each $n$ (though since the MLE is biased, you might get unexpected results) $\endgroup$
    – Henry
    May 3, 2020 at 14:16
  • $\begingroup$ ok I see! Thanks $\endgroup$ May 3, 2020 at 19:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .