# MLE, Confidence Interval, and Asymptotic Distributions

The Question:

Suppose we have i.i.d. random variables $X_1, \dots, X_n$ that have distribution $N(\mu,\sigma ^2)$, where $\mu$ is known and $\sigma$ is a parameter. Let $\psi = \ln \sigma$.

Construct an approximate $95$% confidence interval for $\psi$, and explain how it can be used to find an approximate $95$% confidence interval of $\sigma$.

What I Have Tried:

First, I found the MLE of $\sigma$ to be $$\hat \sigma = \sqrt{\frac 1n \sum_{i=1}^{n}(X_i-\mu)^2}$$

And then I found the asymptotic normal approximation for the distribution of $\hat \sigma$ to be $$\hat \sigma \approx N(\sigma, \frac{\sigma^2}{2n})$$

Applying the delta method, I found the asymptotic distribution of $\hat \psi$ to be

$$\hat \psi \approx N \biggl ( \ln \sigma, \frac{1}{2n} \biggl)$$

(Is this correct? See below for full calculations) It follows that $\frac{1}{\sqrt {2n}}(\hat \psi - \ln \sigma) \sim N(0,1)$, so the $95$% confidence interval is

\begin{align} \ & \Bbb P (-1.96<\frac{1}{\sqrt {2n}}(\psi - \ln \sigma))<1.96 \approx 0.95 \\ \ \implies & \Bbb P \bigl(-1.96\sqrt {2n}+ \ln \sigma<\psi<1.96\sqrt {2n}+ \ln \sigma \bigl) \approx 0.95 \\ \end{align}

Now here is the problem. When I substitute back $\psi = \ln \sigma$, I get \begin{align} \ & \Bbb P \bigl(-1.96\sqrt {2n}+ \ln \sigma<\ln \sigma<1.96\sqrt {2n}+ \ln \sigma \bigl) \approx 0.95 \\ \ \implies & \Bbb P \bigl(-1.96\sqrt {2n}< 0 <1.96\sqrt {2n} \bigl) \approx 0.95 \end{align}

...so this does not give me the $95$% confidence interval for $\sigma$. What should I have done? Is it my method that is incorrect? Or have I made some computational error somewhere (see below)?

Any help would much appreciated, thanks!

$$\sim$$

I defined the new set of i.i.d. random variables $Y_1,\dots,Y_n$ where $Y_i=(X_i-\mu)^2$, and the function $g(u)=\ln (\sqrt u) = \frac 12 \ln (u)$. With this, we can write $$\hat \psi = \ln \hat \sigma = \ln \sqrt{\frac 1n \sum_{i=1}^{n}(X_i-\mu)^2} = \frac 12 \ln \biggl [\frac 1n \sum_{i=1}^{n}(X_i-\mu)^2 \biggl ]= g(\bar Y)$$ where $\bar Y=\frac 1n \sum_{i=1}^n Y_i$.

Calculating the mean and variance of the $Y_i$:

$$\Bbb E (Y_i)=\Bbb E ((X_i-\mu)^2)=\Bbb E ((X_i-\Bbb E (X_i))^2)=Var(X_i)=\sigma^2$$

\begin{align} \ \Bbb E(Y_i^2)&=\Bbb E ((X_i-\mu)^4) \\ \ & =\int_{- \infty}^{\infty}\frac{x^4}{\sqrt{2 \pi \sigma^2}}e^{-x^2/2\sigma^2}dx \\ \ & = \biggl [x^3\frac{-\sigma^2}{\sqrt{2 \pi \sigma^2}}e^{-x^2/2\sigma^2} \biggl ]_{-\infty}^{\infty}+\int_{- \infty}^{\infty}\frac{3x^2 \sigma ^2}{\sqrt{2 \pi \sigma^2}}e^{-x^2/2\sigma^2}dx \\ \ & = [0]+3\sigma^2 Var(X_i-\mu) \\ \ & = 3\sigma^4 \\ \end{align} So $$Var(Y_i)=\Bbb E (Y_i^2)-(\Bbb E (Y_i))^2 = 3\sigma^4 - \sigma^4=2\sigma^4$$

Thus, by the delta method, we have that the asymptotic distribution of $\hat \psi$ is

\begin{align} \ \hat \psi & \approx N \biggl (g(\Bbb E (Y_i)),\frac{g'(\Bbb E (Y_i))^2Var(Y_i)}{n} \biggl) \\ \ & = N\biggl (\ln \sigma, \frac{\frac{1}{4\sigma^4}\cdot 2\sigma^4}{n} \biggl )\\ \ & = N \biggl ( \ln \sigma, \frac{1}{2n} \biggl) \\ \end{align}

You found both asymptotic distribution right: $\hat \psi \approx N \biggl ( \ln \sigma, \frac{1}{2n} \biggl)$ and $\hat \psi \approx N \biggl ( \ln \sigma, \frac{1}{2n} \biggl)$. And the next conclusions you made are wrong. To have the second asymptotic distribution means that $$\sqrt{2n}\left( \hat \psi - \ln \sigma\right)=\sqrt{2n}\left( \hat \psi - \psi\right) \xrightarrow{d} N(0, 1),$$ where $\xrightarrow{d}$ denotes convergence in distribution. Note that $\sqrt{2n}$ is a multiplier here.
Next, $$\mathbb P \left(-1.96<\sqrt {2n}(\hat\psi - \psi))<1.96\right) \to 0.95 \text{ as } n\to\infty.$$ And the asymptotic confidence interval for $\psi$ is $$\hat \psi - \frac{1.96}{\sqrt{2n}} < \psi < \hat \psi + \frac{1.96}{\sqrt{2n}}$$ To obtain the asymptotic CI for $\sigma$, one need to solve the above inequality $$\hat \psi - \frac{1.96}{\sqrt{2n}} < \ln\sigma < \hat \psi + \frac{1.96}{\sqrt{2n}}$$ with respect to $\sigma$. Finally, we obtain $$e^{\hat \psi - \frac{1.96}{\sqrt{2n}}} < \sigma < e^{\hat \psi + \frac{1.96}{\sqrt{2n}}}.$$