2
$\begingroup$

The MLE estimator of the variance of a normal distribution is $\hat \sigma^2 = \frac{1}{N} \sum_{i=1}^{N}(X_i - \bar X)^2$, where $\bar X$ is the sample mean and $X_i \sim^{iid} \mathcal{N}(\mu,\sigma^2)$ . I'm curious because I've seen (e.g. http://www.stat.ufl.edu/~winner/sta6208/allreg.pdf , p.20) that the variance of this estimator is equal to $\frac{2\sigma ^4}{N}$, but I find something different:

Since $\frac{1}{\sigma ^2} \sum_{i=1}^{N}(X_i-\bar X)^2 \sim \chi^2_{N-1}$, we have that

\begin{align} var\big(\frac{N\hat \sigma^2}{\sigma^2}\big) = var(\chi^2_{N-1}) = 2(N-1) \\ \implies var(\hat \sigma ^2) = \frac{2(N-1) \sigma^4}{N^2} \end{align}

What am I missing?

$\endgroup$
2
$\begingroup$

The variance of the estimator in the course notes is based on maximum likelihood estimation which typically results in biased estimators. The second variance calculation has a "correction" term that makes the estimator unbiased. You have likely seen this phenomenon with the unbiased estimator for the sample mean, i.e., dividing by $n-1$ instead of $n$.

There are lots of different ways to generate estimators and the resulting estimators will have different properties. One property that many people like in their estimators is for them to be unbiased. However, people are sometimes willing to accept a little bias to reduce variance. If you interested in this topic you might want to look up bias-variance tradeoff.

$\endgroup$
  • $\begingroup$ So, if I don't divide by $n-1$, the bias is not 0 and the variance will be $\frac{2(n−1)\sigma^4}{n^2}$, so the bias and the variance are greater than in the case where I divide by $n-1$? $\endgroup$ – Amanjo Sep 23 '17 at 1:54
  • $\begingroup$ The estimators themselves are random and have distributions. The bias of an estimator quantifies how far its mean over all samples (usually of a particular size, e.g., $n$) in the population is from the (true/actual but unknown) value of the parameter you are estimating. Similarly, an estimator has its own variance which roughly conveys how far (on average) an estimate based on a particular sample can be from the value of the parameter you are estimating. It might be confusing because you are estimating a variance and both estimators (notes & yours) of the variance have their own variances. $\endgroup$ – shoda Sep 23 '17 at 2:01
  • $\begingroup$ I got it. The variance in the no bias case (i.e. dividing by $n-1$) is greater than in the bias case (i.e. dividing by $n$). $\endgroup$ – Amanjo Sep 23 '17 at 2:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.