4
$\begingroup$

This question is from George Casella statistical inference textbook 10.35 (b).

Let $X_1,...,X_n$ be a random sample from a $n(\mu,\sigma^2)$ population. If $\sigma^2$ is unknown and $\mu$ is known, find a Wald statistic for testing $H_0: \sigma =\sigma_0$.

My attempt: I refer to textbook page 493. First, I need to find the MLE of $\sigma^2$. I find it and is the same as the solution. The log likelihood is

$$-\frac{n}{2}log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\Sigma(x_i-\mu)^2$$. Then taking derivatives with $\sigma^2$, I get the MLE of $\sigma^2$ is $\frac{\Sigma(x_i-\mu)^2}{n}$. Then I got problems. According to page 473, next I need to get the approximate variance of the estimator. First, I need to get the observed information number. Just taking second derivatives and then adding a minus sign).

My frist derivative w.r.t $\sigma$ is: $$-\frac{n}{\sigma}+\frac{\Sigma(x_i-\mu)^2}{\sigma^3}$$

My second derivative w.r.t $\sigma$ is: $$\frac{n}{\sigma^2}-\frac{3\Sigma(x_i-\mu)^2}{\sigma^4}$$ Add a minus sign:$$-\frac{n}{\sigma^2}+\frac{3\Sigma(x_i-\mu)^2}{\sigma^4}$$ Now plug in $\sigma=\sqrt{\frac{\Sigma(x_i-\mu)^2}{n}}$, I got the observed information number is $\frac{2n^2}{\Sigma(x_i-\mu)^2}$. Hence, according to page 473 formula, my variance of the MLE of $\sigma$ is the reciprocal, that is $\frac{\Sigma(x_i-\mu)^2}{2n^2}$. Hence compared with the. solution, our numerator is the same. But my denominator is $\sqrt{\frac{\Sigma(x_i-\mu)^2}{2n^2}}$, different from the solution. What step did I get wrong?

The solution is here: enter image description here

One comment below said I cannot take derivatives r.p.t $\sigma$. I don't know why. I think it's okay for me to do this, because my goal is to get the approximate variance of $\sigma^2$.

But if I choose to take derivatives r.p.t $\sigma^2$ as suggested, then my first derivative of log likelihood is

$$-\frac{n}{2\sigma^2}+\frac{\Sigma(x_i-\mu)^2}{2(\sigma^2)^2}$$. My second derivative is

$$\frac{n}{2(\sigma^2)^2}-\frac{\Sigma(x_i-\mu)^2}{(\sigma^2)^3}$$ Then add a minus sign is:

$$-\frac{n}{2(\sigma^2)^2}+\frac{\Sigma(x_i-\mu)^2}{(\sigma^2)^3}$$

Now plug in MLE $\sigma^2=\frac{\Sigma(x_i-\mu)^2}{n}$, I got the observed information number is $\frac{n^3}{2(\Sigma(x_i-\mu)^2)^2}$. Now it is the same as the solution.

But I am still confused why my previous method to take derivatives r.p.t $\sigma$ failed. My idea is if I take derivatives r.p.t $\sigma$ directly, then I don't need to use delta method to get the variance of $\sigma$ from the variance of of $\sigma^2$

Also, I am stuck how to get the next. According to the solution, now I should use delta method to get the variance of $\sigma$.

My preferred version of delta method is if $W_n \sim AN(a, b_n)$, where $b_n$ goes to 0, and g is differentiable with $g'(a)$ not 0, then $g(W_n) \sim AN(g(a), [g'(a)]^2 b_n)$. (AN denotes approximate normal).

So my $W_n$ is $\frac{\Sigma(x_i-\mu)^2}{n}$, my $a$ is $\sigma^2$, my $b_n$ is $\frac{2(\Sigma(x_i-\mu)^2)^2}{n^3}$. My g is $\sqrt{}$. Hence, my approximated variance of $\sigma$ is

$$[g'(a)]^2 b_n=(\frac{1}{2\sqrt{a}})^2 b_n=\frac{1}{4a} b_n=\frac{1}{4\sigma^2} \frac{2(\Sigma(x_i-\mu)^2)^2}{n^3}$$.

It's weird. My approximated variance of $\sigma$ contains $\sigma^2$. Something in my attempt is wrong here.

$\endgroup$
8
  • $\begingroup$ You took two derivatives with respect the the variable $\sigma$ when you should be taking two derivatives with respect to $\sigma^2$. Note $$\frac{d}{d (\sigma^2)}\Bigg[-\frac{n}{2} \log(2\pi \sigma^2)-\frac{1}{2 \sigma^2}\sum{(X_i - \mu)^2}\Bigg]=-\frac{n}{2 \sigma^2}+\frac{1}{2(\sigma^2)^2}\sum(X_i-\mu)^2$$ This is very different than your result of $-\frac{n}{\sigma}+\frac{\sum(X_i- \mu)^2}{\sigma^3}$. If you keep this in mind throughout your calculation you'll get the desired result. $\endgroup$
    – Matthew H.
    Apr 7, 2021 at 4:47
  • $\begingroup$ I don't know why I cannot take derivatives r.p.t $\sigma$. I think I can since my goal is to get the approximate variance of $\sigma$. $\endgroup$
    – Mariana
    Apr 7, 2021 at 12:58
  • $\begingroup$ But I do this ( taking two derivatives with respect to 𝜎2) It's still not correct (different from the solution.) I added in my question. $\endgroup$
    – Mariana
    Apr 7, 2021 at 13:13
  • $\begingroup$ How is that different? The expression you got equals $\frac{n}{2 \hat{\sigma} _{\mu}^2}$ which is the same information number in the picture posted. $\endgroup$
    – Matthew H.
    Apr 7, 2021 at 13:49
  • 1
    $\begingroup$ Procedures for estimating the variance of a statistic are not unique, and different procedures may yield different estimations. Your way may be perfectly legitimate. The author just chose to first estimate the variance of $\hat{\sigma}^2$ and then applied the delta method (taking $g(x)=\sqrt{x}$) to estimate the variance of $\hat{\sigma}$. $\endgroup$
    – Matthew H.
    Apr 7, 2021 at 14:42

1 Answer 1

7
$\begingroup$

It's official. I disagree with the solution you have provided. Let me share my perspective on this problem.

The Fisher information $\mathcal{I}(\sigma^2)$ can be expressed as $$\mathcal{I}(\sigma^2)=\mathbb{E}\Bigg(-\frac{\partial ^2l(X_1,\ldots,X_n;\sigma^2)}{\partial (\sigma^2)^2}\Bigg|\sigma^2\Bigg)$$ where $l(x_1,,\ldots,x_n;\sigma^2)$ is your "log$-$likelihood" $$l(x_1,,\ldots,x_n;\sigma^2)=-\frac{n}{2}\ln(2\pi \sigma^2)-\frac{1}{2\sigma^2}\sum_{i=1}^n(x_i-\mu)^2$$ You accurately found $-\frac{\partial ^2l(X_1,\ldots,X_n;\sigma^2)}{\partial (\sigma^2)^2}$ to be $-\frac{n}{2(\sigma^2)^2}+\frac{\sum_{i=1}^n(x_i - \mu)^2}{(\sigma^2)^3}$ which we can express as $$-\frac{n}{2\sigma^4}+\frac{1}{\sigma^4}\sum_{i=1}^n\Big(\frac{x_i - \mu}{\sigma}\Big)^2$$ Since $\sum_{i=1}^n\Big(\frac{X_i - \mu}{\sigma}\Big)^2$ given $\sigma^2$ possesses a $\chi^2$ distribution with $n$ degrees of freedom, we compute the Fisher information $\mathcal{I}(\sigma^2)$ to $$\mathcal{I}(\sigma^2)=-\frac{n}{2\sigma^4}+\frac{1}{\sigma^4}\mathbb{E}\Bigg(\sum_{i=1}^n\Big(\frac{X_i - \mu}{\sigma}\Big)^2\Bigg|\sigma^2\Bigg)=-\frac{n}{2\sigma^4}+\frac{n}{\sigma^4}=\frac{n}{2\sigma^4}$$ Evidently, the reciprocal of $\mathcal{I}(\sigma^2)$ is used as an estimator for the variance of the MLE of $\sigma^2,$ which in this case equals $\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(X_i-\mu)^2$. What's even more interesting is that $\frac{2\sigma^4}{n}$ is the exact variance of $\hat{\sigma}^2|\sigma^2$. To see this, first write $\hat{\sigma}^2=\frac{\sigma^2}{n}\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2$ and again noting that $$\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2\Big|\sigma^2 \sim \chi^2_{n}$$ it follows $$\mathbb{V}(\hat{\sigma}^2|\sigma^2)=\Big(\frac{\sigma^2}{n}\Big)^2\cdot \mathbb{V}\Bigg(\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2\Bigg|\sigma^2\Bigg)=\Big(\frac{\sigma^2}{n}\Big)^2\cdot 2n=\frac{2 \sigma^4}{n}$$ If we assume $H_0$ is true i.e. that $\sigma= \sigma_0$ then by CLT our MLE $\hat{\sigma}^2$ is asymptotically $N(\sigma_0^2,\frac{2\sigma_0^4}{n})$. This means $$\sqrt{n}\big(\hat{\sigma}^2-\sigma_0^2\big)\approx N\Big(0,(\sigma_0^2 \sqrt{2})^2\Big)$$ Applying the $\delta-$ method with $g(x)=\sqrt{x}$ yields $$\sqrt{n}\big(\hat{\sigma}-\sigma_0\big)=\sqrt{n}\big(g(\hat{\sigma}^2)-g(\sigma_0^2)\big)\approx N\Big(0,(\sigma_0^2 g'(\sigma_0^2) \sqrt{2})^2\Big)$$ The above is equivalent to $\sqrt{n}\Big(\hat{\sigma}-\sigma_0\Big)\approx N(0,\sigma_0^2/2)$ and an appropaite Wald statistic would be $$\frac{\hat{\sigma}-\sigma_0}{\sqrt{\sigma_0^2/2n}}$$

$\endgroup$
2
  • 3
    $\begingroup$ It is straightforward to simulate the variance of the sample standard deviation in Mathematica; e.g., d[s_, n_, m_] := ParallelTable[Sqrt[Total[RandomVariate[NormalDistribution[0, s], n]^2]/n], m] computes a list of $m$ realizations of the sample standard deviation from a normal distribution with mean $0$ and variance $s^2$ with sample size $n$. Then Variance[d[10, 25, 10^5]] should give a number that is approximately $10^2/(2(25)) = 2$. The official solution's factor of $8$ in the denominator is incorrect. $\endgroup$
    – heropup
    Apr 7, 2021 at 22:37
  • $\begingroup$ I ask a follow-up question here: math.stackexchange.com/questions/4096288/… $\endgroup$
    – Mariana
    Apr 10, 2021 at 0:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .