5
$\begingroup$

EDIT: I have followed up to this discussion with a second question: https://math.stackexchange.com/questions/635567/hypothesis-test-on-variance-of-normal-sample

I am preparing for a stat exam and I was trying to derive the distribution of the likelihood ratio statistic for the hypothesis test below.

Let $X_1 ... X_{n}$ be a random sample from a $N(\mu,\sigma^2)$ distribution, where $\mu$ is known and $\sigma^2$ is unknown. I want to test the hypothesis $H_0 : \sigma^2 = \sigma_{0}^{2} $ vs. $H_1 : \sigma^2 \neq \sigma_{0}^{2}$ (and, trivially, $\sigma^2 >0$).

The generic joint pdf for the n independent random variables (ie. the likelihood function for the random sample) is:

$L=\prod_{i=1}^{n} \large(\frac{1}{\sqrt{2\pi\sigma^2}})\cdot e^-\frac{(X_i - \mu)^2}{2\sigma^2}= \large(\frac{1}{\sqrt{2\pi\sigma^2}})^{n}\cdot e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma^2}$

Under the null hypothesis, the maximum value taken by $L$ is: $\large(\frac{1}{\sqrt{2\pi\sigma_{0}^{2}}})^{n}\cdot e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}$

If we do not constrain $\sigma^{2}$ to be equal to $\sigma_{0}^{2}$, then $L$ is maximised by deriving the maximum likelihood estimator for $\sigma^{2}$, ie $\hat{\sigma}^{2}=\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{n}$

In this case, the maximum likelihood becomes: $\large(\frac{1}{\sqrt{2\pi\hat{\sigma_{0}}^{2}}})^{n}\cdot e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\hat{\sigma_{0}}^{2}}$

Setting these as numerator and denominator, respectively, I get the following likelihood ratio statistic

$\Lambda = \LARGE\frac{\large(\frac{1}{\sqrt{2\pi\sigma_{0}^{2}}})^{n}\cdot e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}}{\large(\frac{1}{\sqrt{2\pi\hat{\sigma_{0}}^{2}}})^{n}\cdot e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\hat{\sigma_{0}}^{2}}} \\ = \large(\frac{\hat{\sigma_{0}}^{2}}{\sigma_{0}^{2}})^{n/2}\cdot {e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}}\cdot{e^\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\hat{\sigma_{0}^{2}}}} \\ = \large(\frac{\hat{\sigma_{0}}^{2}}{\sigma_{0}^{2}})^{n/2}\cdot {e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}}\cdot{e^{\sum_{i=1}^{n} (X_i - \mu)^2\cdot0.5\cdot\frac{n}{\sum_{i=1}^{n} (X_i - \mu)^2}}} \\ = \large(\frac{\hat{\sigma_{0}}^{2}}{\sigma_{0}^{2}})^{n/2}\cdot {e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}}\cdot{e^{0.5}}$

Since the test corresponds to $\Lambda \leq k$ for some constant k, we can write:

$\Lambda= \large(\frac{\hat{\sigma_{0}}^{2}}{\sigma_{0}^{2}})^{n/2}\cdot {e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}}\cdot{e^{0.5}} \leq k$

And hence: $\large(\frac{\hat{\sigma_{0}}^{2}}{\sigma_{0}^{2}})^{n/2}\cdot {e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}} \leq k'$

When I get here, there are two things I do not understand:

  • I do not think that you can also bring ${e^-\frac{\sum_{i=1}^{n} (X_i - \mu)^2}{2\sigma_{0}^{2}}}$ to the right side as this is a function of the random sample. Am I missing something?
  • If the previous statement is correct, how do you find the distribution of the left hand side?

Any clarification, link or reference would be extremely helpful.

$\endgroup$

1 Answer 1

3
$\begingroup$

As you yourself write, the maximized likelihood given the sample is

$$L(\hat{\sigma}^{2} \mid \mathbf x) = \left(\frac{1}{\sqrt{2\pi\hat{\sigma}^{2}}}\right)^{n}\cdot e^-\frac{\sum_{i=1}^{n} (x_i - \mu)^2}{2\hat{\sigma}^{2}}$$

and you have that

$$\hat{\sigma}^{2}=\frac{\sum_{i=1}^{n} (x_i - \mu)^2}{n}$$

Inserting this into the likelihood we get

$$L(\hat{\sigma}^{2} \mid \mathbf x) = \left(\frac{1}{\sqrt{2\pi\hat{\sigma}^{2}}}\right)^{n}\cdot e^{-(n/2)} $$

Then the Likelihood Ratio is

$$\Lambda = \frac{ \left(\frac{1}{\sqrt{2\pi\sigma_0^2}}\right)^{n/2}\cdot e^{-\frac{\sum_{i=1}^n (X_i - \mu)^2}{2\sigma_0^2}}} {\left(\frac{1}{\sqrt{2\pi\hat{\sigma}^{2}}}\right)^{n}\cdot e^{-(n/2)}} = \left(\frac {\hat{\sigma}^2}{\sigma_0^2}\right)^{n/2} \cdot \exp\left \{-\frac 12\left(\frac{\sum_{i=1}^n (X_i - \mu)^2}{\sigma_0^2}-n\right)\right\}$$

and using again the expression for $\hat \sigma^2$ we get

$$\Lambda = \left(\frac {\hat{\sigma}^2}{\sigma_0^2}\right)^{n/2} \cdot \exp\left \{-\frac n2\left(\frac {\hat{\sigma}^2}{\sigma_0^2}-1\right)\right\}$$

Now, under the null, the random variable denoted

$$z_i^2 = \left(\frac {x_i - \mu}{\sigma_0}\right)^2 \sim \chi^2(1)$$

We have

$$ \frac {\hat{\sigma}^2}{\sigma_0^2} = \frac 1n\sum_{i=1}^{n} \left(\frac {x_i - \mu}{\sigma_0}\right)^2 = \frac 1n \sum_{i=1}^{n}z_i^2$$

So we can write $$\Lambda = \left(\frac 1n \sum_{i=1}^{n}z_i^2 \right)^{n/2} \cdot \exp\left \{-\frac n2\left( \frac 1n \sum_{i=1}^{n}z_i^2-1\right)\right\}$$

Taking minus log we have

$$-\ln \Lambda = -\frac n2 \ln \left(\frac 1n \sum_{i=1}^{n}z_i^2 \right) +\frac n2\left( \frac 1n \sum_{i=1}^{n}z_i^2-1\right)$$

Manipulating the second term in the RHS,

$$= -\frac n2 \ln \left(\frac 1n \sum_{i=1}^{n}z_i^2 \right) + \sqrt {\frac n 2} \left( \frac {\sum_{i=1}^{n}(z_i^2-1)}{\sqrt {2n}}\right)$$ and multiplying throughout by $\sqrt {\frac 2n}$ we obtain

$$-\sqrt {\frac 2n} \ln \Lambda = -\sqrt {\frac n 2} \ln \left(\frac 1n \sum_{i=1}^{n}z_i^2 \right) + \left( \frac {\sum_{i=1}^{n}(z_i^2-1)}{\sqrt {2n}}\right)$$

The second term in the RHS is a standardized sum of i.i.d $\chi^2(1)$ random variables, each having mean equal to $1$, variance equal to $2$, and so the standard deviation of the sum is $\sqrt {2n}$. This quantity will converge to a $N(0,1)$. Then (abusing notation a bit),

$$\operatorname{plim}\left(-\sqrt {\frac 2n} \ln \Lambda\right) = \operatorname{plim}\left(-\sqrt {\frac n 2} \right)\cdot \operatorname{plim}\left[\ln \left(\frac 1n \sum_{i=1}^{n}z_i^2 \right)\right] + \operatorname{plim}\left( \frac {\sum_{i=1}^{n}(z_i^2-1)}{\sqrt {2n}}\right) $$

$$= \infty \cdot \left[\ln \left(\operatorname{plim}\frac 1n \sum_{i=1}^{n}z_i^2 \right)\right] + N(0,1)=\infty \cdot \ln (1) + N(0,1) = 0 + N(0,1)$$

(accepting $\infty \cdot 0 =0$)

which means that the quantity

$$Q = -\sqrt {\frac 2n} \ln \Lambda \rightarrow_d N(0,1)$$

if the null hypothesis is true. For a finite sample this quantity is calculated as

$$\hat Q(\hat\sigma^2,\sigma_0^2,n) = \sqrt {\frac n2}\left[\frac {\hat{\sigma}^2}{\sigma_0^2}-1-\ln \left(\frac {\hat{\sigma}^2}{\sigma_0^2}\right)\right]$$

$\endgroup$
9
  • 2
    $\begingroup$ 1. The wikipedia article writes the same thing as I do, because $(\chi^2_k-k) = \sum_k\chi^2_1 -k = \sum_k(\chi^2_1 -1)$. 2. No, I was just paying my dues to mathematical rigor. Note also at what rate the one term goes to infinity compared to the rate the other goes to unity and hence its log to zero. 3. As can be inferred from my answer, and the properties of the chi-square distribution, the ratio of the sample variance over $\sigma^2_0$ has, under the null, an exact finite-sample Gamma distribution with shape parameter $k=n/2$ and scale parameter $\theta = 2/n$. This means ...(CONTD) $\endgroup$ Jan 12, 2014 at 11:36
  • 2
    $\begingroup$ (CONTD)... that it has an expected value equal to $1$ and variance equal to $2/n$. You might want to take it from here. $\endgroup$ Jan 12, 2014 at 11:37
  • 2
    $\begingroup$ I am not sure I am following you, perhaps it is because I am rather tired at the moment. On point 1, the variance of the sum is $var(\sum_k\chi^2_1) = Var (\chi^2_1)+...+Var (\chi^2_1) = 2+...+2$, $n$ times, so $var(\sum_k\chi^2_1) = 2n$. On point 2, $\left(\frac 1n \sum_{i=1}^{n}z_i^2 \right)^\sqrt{n}$ goes to $1$ and so its logarithm goes to zero. Am I missing something? $\endgroup$ Jan 12, 2014 at 12:51
  • 2
    $\begingroup$ Ignore the part of the comment referring to point 2. It just shows that indeed, the expression $0\cdot \infty$ is formally indeterminate. Nevertheless it is indeed treated as zero, and I have somewhere a discussion on the matter -when I find it I will post it. $\endgroup$ Jan 12, 2014 at 13:03
  • 2
    $\begingroup$ Regarding Point 1, to see that you must divide by the standard deviation of the whole sum, and not by the standard deviation of one of the i.i.d rv, consider what you propose: it is the expression $\frac {\sum_{i=1}^{n}(z_i^2-1)}{\sqrt {2}}$ or in general terms $\sum_{i=1}^{n}\frac {x_i-\mu}{\sigma}$, where $\mu$ and $\sigma$ are the mean and standard deviation respectively. This sum therefore is a sum of $n$ standardized random variables. (CONTD)... $\endgroup$ Jan 13, 2014 at 8:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .