1
$\begingroup$

Consider $N(\mu,\sigma^2)$ and pick independent $X_1, \cdots, X_n$.

It is a well known fact that the MLE estimator $$ S_n^2= \frac 1 n\sum(X_i - \overline X_n)^2 $$ is a biased estimator for $\sigma^2$.

If we suppose $\mu= 0$, then I get it is unbiased. Am I wrong? Thanks!

$\endgroup$
2
$\begingroup$

If you somehow know the value of $\mu,$ regardless of whether it's $0$ or not, then you should use that instead of the estimate $\bar X_n,$ and in that case the MLE of $\sigma^2$ is $\displaystyle \frac 1 n \sum_{i=1}^n (X_i-\mu)^2,$ and that is unbiased. To see that it's unbiased, just observe that $\operatorname E((X_i-\mu)^2) = \sigma^2.$

But $\displaystyle \frac 1 n \sum_{i=1}^n (X_i-\bar X)^2 $ still has expected value $\frac{n-1} n \sigma^2.$ If $\mu$ is known, then that estimator is still biased but is not the MLE.

To say that $\mu$ is “known” simply means the family of distributions you're working with is $\left\{ N(\mu,\sigma^2): \sigma^2>0 \right\}$ rather than $\left\{ N(\mu,\sigma^2) : \sigma^2>0\ \&\ \mu\in\mathbb R \right\}.$ The likelihood function is then a function of $\sigma$ or of $\sigma^2$ rather than of two variables, one of which is $\mu.$

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

Note that if we know the mean $\mu$ then $$E[S_n^2] = \frac{1}{n}\sum_{i=1}^{n}E[(X_i-\mu)^2]=\sigma^2.$$ That is, $S_n^2$ is an unbiased estimate of $\sigma^2$.

On the other hand, if we estimate the mean by using the sample average and compute $$S_n^2 = \frac{1}{n}\sum_{i=1}^{n}(X_i-\bar{X})^2,$$ then it no longer remains unbiased.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

If $\mu = 0$ then $\sum_{i=1}^n X_i^2/\sigma^2 \sim \chi^2(n)$, hence $$ \mathbb{E}(\frac{1}{n}\sum_{i=1}^n X_i^2) = \mathbb{E}(\frac{\sigma^2}{n} \sum_{i=1}^n X_i^2/\sigma^2) = \sigma ^ 2\frac{n}{n} = \sigma^2. $$ This is true for any other (true) value of $\mu$. Where if you estimate $\mu$ with $\bar{X}$ then $\sum_{i=1}(X_i - \bar{X})^2/\sigma^2 \sim \chi^2_{(n-1)}$ because $$ \frac{\sum_{i=1}(X_i - \bar{X})^2}{\sigma^2} = \frac{\sum_{i=1}(X_i - \mu)^2}{\sigma^2} - \frac{n(\bar{X} - \mu)^2}{\sigma^2} = \chi^2_{(n)} - \chi^2_{(1)} = \chi^2_{(n-1)}, $$ where the last equality holds because in normal distribution, $\sum_{i=1}^n (X_i - \mu)^2$ is independent of $\bar{X}_n$.
As such, dividing by $n$ instead of the actual degrees of freedom $(n-1)$ will result in a bias.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ In order that $ \chi^2_{(n)} - \chi^2_{(1)} = \chi^2_{(n-1)}$ make sense, you need to assume $\text{“} \chi^2_{(1)}\text{''}$ and $\text{“} \chi^2_{(n-1)} \text{''}$ are independent of each other and that neither of those is independent of $\text{“} \chi^2_{(n)} \text{''}. \qquad$ $\endgroup$ – Michael Hardy Mar 18 '18 at 16:30
  • $\begingroup$ Well, to the best of my knowledge, the sample is from normal distribution, so the independence is inherent... $\endgroup$ – V. Vancak Mar 18 '18 at 19:11
  • $\begingroup$ The independence can be deduced from the assumptions. But which ones are independent and which are not could at least be mentioned. $\endgroup$ – Michael Hardy Mar 18 '18 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.