Ask a question on Wald statistic (George Casella 10.35 (b))

This question is from George Casella statistical inference textbook 10.35 (b).

Let $$X_1,...,X_n$$ be a random sample from a $$n(\mu,\sigma^2)$$ population. If $$\sigma^2$$ is unknown and $$\mu$$ is known, find a Wald statistic for testing $$H_0: \sigma =\sigma_0$$.

My attempt: I refer to textbook page 493. First, I need to find the MLE of $$\sigma^2$$. I find it and is the same as the solution. The log likelihood is

$$-\frac{n}{2}log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\Sigma(x_i-\mu)^2$$. Then taking derivatives with $$\sigma^2$$, I get the MLE of $$\sigma^2$$ is $$\frac{\Sigma(x_i-\mu)^2}{n}$$. Then I got problems. According to page 473, next I need to get the approximate variance of the estimator. First, I need to get the observed information number. Just taking second derivatives and then adding a minus sign).

My frist derivative w.r.t $$\sigma$$ is: $$-\frac{n}{\sigma}+\frac{\Sigma(x_i-\mu)^2}{\sigma^3}$$

My second derivative w.r.t $$\sigma$$ is: $$\frac{n}{\sigma^2}-\frac{3\Sigma(x_i-\mu)^2}{\sigma^4}$$ Add a minus sign:$$-\frac{n}{\sigma^2}+\frac{3\Sigma(x_i-\mu)^2}{\sigma^4}$$ Now plug in $$\sigma=\sqrt{\frac{\Sigma(x_i-\mu)^2}{n}}$$, I got the observed information number is $$\frac{2n^2}{\Sigma(x_i-\mu)^2}$$. Hence, according to page 473 formula, my variance of the MLE of $$\sigma$$ is the reciprocal, that is $$\frac{\Sigma(x_i-\mu)^2}{2n^2}$$. Hence compared with the. solution, our numerator is the same. But my denominator is $$\sqrt{\frac{\Sigma(x_i-\mu)^2}{2n^2}}$$, different from the solution. What step did I get wrong?

The solution is here:

One comment below said I cannot take derivatives r.p.t $$\sigma$$. I don't know why. I think it's okay for me to do this, because my goal is to get the approximate variance of $$\sigma^2$$.

But if I choose to take derivatives r.p.t $$\sigma^2$$ as suggested, then my first derivative of log likelihood is

$$-\frac{n}{2\sigma^2}+\frac{\Sigma(x_i-\mu)^2}{2(\sigma^2)^2}$$. My second derivative is

$$\frac{n}{2(\sigma^2)^2}-\frac{\Sigma(x_i-\mu)^2}{(\sigma^2)^3}$$ Then add a minus sign is:

$$-\frac{n}{2(\sigma^2)^2}+\frac{\Sigma(x_i-\mu)^2}{(\sigma^2)^3}$$

Now plug in MLE $$\sigma^2=\frac{\Sigma(x_i-\mu)^2}{n}$$, I got the observed information number is $$\frac{n^3}{2(\Sigma(x_i-\mu)^2)^2}$$. Now it is the same as the solution.

But I am still confused why my previous method to take derivatives r.p.t $$\sigma$$ failed. My idea is if I take derivatives r.p.t $$\sigma$$ directly, then I don't need to use delta method to get the variance of $$\sigma$$ from the variance of of $$\sigma^2$$

Also, I am stuck how to get the next. According to the solution, now I should use delta method to get the variance of $$\sigma$$.

My preferred version of delta method is if $$W_n \sim AN(a, b_n)$$, where $$b_n$$ goes to 0, and g is differentiable with $$g'(a)$$ not 0, then $$g(W_n) \sim AN(g(a), [g'(a)]^2 b_n)$$. (AN denotes approximate normal).

So my $$W_n$$ is $$\frac{\Sigma(x_i-\mu)^2}{n}$$, my $$a$$ is $$\sigma^2$$, my $$b_n$$ is $$\frac{2(\Sigma(x_i-\mu)^2)^2}{n^3}$$. My g is $$\sqrt{}$$. Hence, my approximated variance of $$\sigma$$ is

$$[g'(a)]^2 b_n=(\frac{1}{2\sqrt{a}})^2 b_n=\frac{1}{4a} b_n=\frac{1}{4\sigma^2} \frac{2(\Sigma(x_i-\mu)^2)^2}{n^3}$$.

It's weird. My approximated variance of $$\sigma$$ contains $$\sigma^2$$. Something in my attempt is wrong here.

• You took two derivatives with respect the the variable $\sigma$ when you should be taking two derivatives with respect to $\sigma^2$. Note $$\frac{d}{d (\sigma^2)}\Bigg[-\frac{n}{2} \log(2\pi \sigma^2)-\frac{1}{2 \sigma^2}\sum{(X_i - \mu)^2}\Bigg]=-\frac{n}{2 \sigma^2}+\frac{1}{2(\sigma^2)^2}\sum(X_i-\mu)^2$$ This is very different than your result of $-\frac{n}{\sigma}+\frac{\sum(X_i- \mu)^2}{\sigma^3}$. If you keep this in mind throughout your calculation you'll get the desired result. Apr 7, 2021 at 4:47
• I don't know why I cannot take derivatives r.p.t $\sigma$. I think I can since my goal is to get the approximate variance of $\sigma$. Apr 7, 2021 at 12:58
• But I do this ( taking two derivatives with respect to 𝜎2) It's still not correct (different from the solution.) I added in my question. Apr 7, 2021 at 13:13
• How is that different? The expression you got equals $\frac{n}{2 \hat{\sigma} _{\mu}^2}$ which is the same information number in the picture posted. Apr 7, 2021 at 13:49
• Procedures for estimating the variance of a statistic are not unique, and different procedures may yield different estimations. Your way may be perfectly legitimate. The author just chose to first estimate the variance of $\hat{\sigma}^2$ and then applied the delta method (taking $g(x)=\sqrt{x}$) to estimate the variance of $\hat{\sigma}$. Apr 7, 2021 at 14:42

The Fisher information $$\mathcal{I}(\sigma^2)$$ can be expressed as $$\mathcal{I}(\sigma^2)=\mathbb{E}\Bigg(-\frac{\partial ^2l(X_1,\ldots,X_n;\sigma^2)}{\partial (\sigma^2)^2}\Bigg|\sigma^2\Bigg)$$ where $$l(x_1,,\ldots,x_n;\sigma^2)$$ is your "log$$-$$likelihood" $$l(x_1,,\ldots,x_n;\sigma^2)=-\frac{n}{2}\ln(2\pi \sigma^2)-\frac{1}{2\sigma^2}\sum_{i=1}^n(x_i-\mu)^2$$ You accurately found $$-\frac{\partial ^2l(X_1,\ldots,X_n;\sigma^2)}{\partial (\sigma^2)^2}$$ to be $$-\frac{n}{2(\sigma^2)^2}+\frac{\sum_{i=1}^n(x_i - \mu)^2}{(\sigma^2)^3}$$ which we can express as $$-\frac{n}{2\sigma^4}+\frac{1}{\sigma^4}\sum_{i=1}^n\Big(\frac{x_i - \mu}{\sigma}\Big)^2$$ Since $$\sum_{i=1}^n\Big(\frac{X_i - \mu}{\sigma}\Big)^2$$ given $$\sigma^2$$ possesses a $$\chi^2$$ distribution with $$n$$ degrees of freedom, we compute the Fisher information $$\mathcal{I}(\sigma^2)$$ to $$\mathcal{I}(\sigma^2)=-\frac{n}{2\sigma^4}+\frac{1}{\sigma^4}\mathbb{E}\Bigg(\sum_{i=1}^n\Big(\frac{X_i - \mu}{\sigma}\Big)^2\Bigg|\sigma^2\Bigg)=-\frac{n}{2\sigma^4}+\frac{n}{\sigma^4}=\frac{n}{2\sigma^4}$$ Evidently, the reciprocal of $$\mathcal{I}(\sigma^2)$$ is used as an estimator for the variance of the MLE of $$\sigma^2,$$ which in this case equals $$\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(X_i-\mu)^2$$. What's even more interesting is that $$\frac{2\sigma^4}{n}$$ is the exact variance of $$\hat{\sigma}^2|\sigma^2$$. To see this, first write $$\hat{\sigma}^2=\frac{\sigma^2}{n}\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2$$ and again noting that $$\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2\Big|\sigma^2 \sim \chi^2_{n}$$ it follows $$\mathbb{V}(\hat{\sigma}^2|\sigma^2)=\Big(\frac{\sigma^2}{n}\Big)^2\cdot \mathbb{V}\Bigg(\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2\Bigg|\sigma^2\Bigg)=\Big(\frac{\sigma^2}{n}\Big)^2\cdot 2n=\frac{2 \sigma^4}{n}$$ If we assume $$H_0$$ is true i.e. that $$\sigma= \sigma_0$$ then by CLT our MLE $$\hat{\sigma}^2$$ is asymptotically $$N(\sigma_0^2,\frac{2\sigma_0^4}{n})$$. This means $$\sqrt{n}\big(\hat{\sigma}^2-\sigma_0^2\big)\approx N\Big(0,(\sigma_0^2 \sqrt{2})^2\Big)$$ Applying the $$\delta-$$ method with $$g(x)=\sqrt{x}$$ yields $$\sqrt{n}\big(\hat{\sigma}-\sigma_0\big)=\sqrt{n}\big(g(\hat{\sigma}^2)-g(\sigma_0^2)\big)\approx N\Big(0,(\sigma_0^2 g'(\sigma_0^2) \sqrt{2})^2\Big)$$ The above is equivalent to $$\sqrt{n}\Big(\hat{\sigma}-\sigma_0\Big)\approx N(0,\sigma_0^2/2)$$ and an appropaite Wald statistic would be $$\frac{\hat{\sigma}-\sigma_0}{\sqrt{\sigma_0^2/2n}}$$
• It is straightforward to simulate the variance of the sample standard deviation in Mathematica; e.g., d[s_, n_, m_] := ParallelTable[Sqrt[Total[RandomVariate[NormalDistribution[0, s], n]^2]/n], m] computes a list of $m$ realizations of the sample standard deviation from a normal distribution with mean $0$ and variance $s^2$ with sample size $n$. Then Variance[d[10, 25, 10^5]] should give a number that is approximately $10^2/(2(25)) = 2$. The official solution's factor of $8$ in the denominator is incorrect. Apr 7, 2021 at 22:37