1
$\begingroup$

This is from George Casella textbook question 8.37 (c). Let $X_1,...,X_n$ be a random sample from a $n(\theta, \sigma^2)$ population. Consider testing $H_0:\theta\leq \theta_0$ versus $H_1:\theta> \theta_0$. If $\sigma^2$ is unknown, what is the LRT statistic? The solution is $\lambda=(\hat{\sigma}^2/\hat{\sigma_0}^2)^{n/2}$. I didn't get this.

My attempt: the LRT of denimoator is easier and reached when $\theta=\bar{x}$ and $\sigma^2=\hat{\sigma}^2$ below in the solution. The numerator is $\sigma^2=\hat{\sigma}^2$ and $\theta=\theta_0$.(We don't need to consider when $\lambda=1$ case.) I didn't get the LRT statistic in the solution.

enter image description here

$\endgroup$
3
$\begingroup$

According to Casella & Berger's notation (Definition 8.2.1), the likelihood ratio test statistic for testing $H_0: \theta \in \Theta_0$ versus $H_1: \theta \in \Theta_0^c$ is $$\lambda(\boldsymbol{x}) = \frac{\text{sup}_{\Theta_0}L(\theta \mid \boldsymbol{x})}{\text{sup}_{\Theta}L(\theta \mid \boldsymbol{x})}.$$ A likelihood ratio test (LRT) is any test that has a rejection region of the form $\{\boldsymbol{x}: \lambda(\boldsymbol{x}) \leq c\}$, where $c$ is any number satisfying $0 \leq c \leq 1$.

In the problem of testing $H_0: \theta \leq \theta_0$ versus $H_1: \theta > \theta_0$, where $X_1, \dots, X_n \overset{i.i.d.}{\sim} n(\theta, \sigma^2)$ and $\sigma^2$ is unknown, we have $\Theta_0 = \{(\theta, \sigma^2): \theta \leq \theta_0, \sigma^2 > 0\}$ and $\Theta = \{(\theta, \sigma^2): \theta \in \mathbb{R}, \sigma^2 > 0\}$, and the likelihood is given by $$L(\theta, \sigma^2 \mid \boldsymbol{x}) = \left(\frac{1}{\sqrt{2\pi\sigma^2}}\right)^n \exp\left(-\frac{1}{2\sigma^2}\sum_{i=1}^n(x_i - \theta)^2\right).$$

The supremum in the denominator of $\lambda(\boldsymbol{x})$ is attained at the unrestricted MLE of $(\theta, \sigma^2)$, that is, $\hat{\theta} = \bar{x}$ and $\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \bar{x})^2$ (See Section 4.1 if this is unclear).

The supremum in the numerator of $\lambda(\boldsymbol{x})$ needs to be a bit more careful, since it is "restricted" to the parameter space $\Theta_0$. Specifically, if we observe $\bar{x} > \theta_0$ ("$>$" is more precise than "$\geq$" the solution), then $L(\theta, \sigma^2)$ cannot be maximized at the unrestricted MLE ($\hat{\theta} = \bar{x}, \hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \bar{x})^2$) since $(\bar{x}, \hat{\sigma}^2) \notin \Theta_0$. In this case, the restricted MLE (with respect to $\Theta_0$) of $(\theta, \sigma^2)$ is taken at the boundary of the parameter space, that is, $\hat{\theta}_0 = \theta_0$ and $\hat{\sigma}_0^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \theta_0)^2$. On the other hand, if we observe $\bar{x} \leq \theta_0$, then $L(\theta, \sigma^2)$ can be maximized at the unrestricted MLE ($\hat{\theta} = \bar{x}, \hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n(x_i - \bar{x})^2$) since $(\bar{x}, \hat{\sigma}^2) \in \Theta_0$. In this case, the numerator and denominator coincides thus $\lambda(\boldsymbol{x}) = 1$.

To sum up, for $n(\theta, \sigma^2)$ where $\sigma^2$ is unknown, the LRT statistic of testing $H_0: \theta \leq \theta_0$ versus $H_1: \theta > \theta_0$ is given by $$\lambda(\boldsymbol{x}) = \begin{cases} \dfrac{\text{sup}_{\Theta_0}L(\theta, \sigma^2 \mid \boldsymbol{x})}{\text{sup}_{\Theta}L(\theta, \sigma^2 \mid \boldsymbol{x})} = \dfrac{L(\theta_0, \hat{\sigma}_0^2 \mid \boldsymbol{x})}{L(\bar{x}, \hat{\sigma}^2 \mid \boldsymbol{x})} = \left(\dfrac{\hat{\sigma}^2}{\hat{\sigma}_0^2}\right)^{n/2} & \text{if } \bar{x} > \theta_0; \\ 1 & \text{if } \bar{x} \leq \theta_0. \end{cases}$$

Remark: It is worth to note the fact that $$\exp\left(-\frac{1}{2\hat{\sigma}_0^2}\sum_{i=1}^n(x_i - \theta_0)^2\right) = \exp\left(-\frac{1}{2\hat{\sigma}^2}\sum_{i=1}^n(x_i - \bar{x})^2\right) = \exp\left(-\frac{n}{2}\right),$$ which cancels in the derivation of the LRT statistic $\lambda(\boldsymbol{x})$ for $\bar{x} > \theta_0$.

$\endgroup$
4
  • 2
    $\begingroup$ Thank you so much! We can also refer to Example 7.2.11. $\endgroup$
    – Mariana
    Apr 4 at 14:27
  • $\begingroup$ I asked another relevant question here. Could you help me? math.stackexchange.com/questions/4089371/… $\endgroup$
    – Mariana
    Apr 4 at 20:09
  • $\begingroup$ @Yichuan No problem! Glad that it is helpful. $\endgroup$
    – X. Li
    Apr 4 at 21:00
  • $\begingroup$ @Yichuan Sure, I can take a look. $\endgroup$
    – X. Li
    Apr 4 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.