I am currently working on an exercise for statistics, but I am stuck at the last part of a question, (rejection regions).

We have $(Y_1, x_1), \dots ,(Y_n, x_n)$, where $\{Y_i\}_i$ are i.i.d. such that $$Y_i \sim \mathcal{N}(\theta x_i, 1), i \in \mathbb{N}.$$

I have shown that the maximum likelihood estimator is $\hat{\theta}_{ML} = \sum_{i = 1}^{n}x_i Y_i / \sum_{i = 1}^{n} x_i^2 $.

Furthermore, I found a $95\%$ confidence interval based on inverting $\hat{\theta}_{ML}$, namely $\left[\hat{\theta}_{ML} - z_{1-\alpha/2}\frac{1}{\sqrt{\sum x_i^2}}, \hat{\theta}_{ML} + z_{1-\alpha/2}\frac{1}{\sqrt{\sum x_i^2}}\right]$. However, I am not sure if this is necessary.

Now we are interested in testing $H_0: \theta = \theta_0$ against $H_1: \theta \neq \theta_0$.

I want to determine the rejection region $[0, c]$ associated with the log likelihood ratio as test statistic. I derived this test statistic, and found that $-2 \log(\lambda(Y)) \sim \chi^2_1$ under $H_0$. i.e., $$(\hat{\theta}_{ML} - \theta_0)^2\sum_{i = 1}^{n} x_i^2 \sim \chi^2_1 $$ under $H_0$.

So I don't really understand the $\chi^2_1$ distribution as yet, and this is where I am stuck. I think that $P($reject $H_0 | \theta = \theta_0) = \alpha$, however, I am quite stuck on how to apply this. Is there someone who could help me a little with this?

Thanks for your help in advance!

  • $\begingroup$ If the $x_i$s are random, what is their distribution? Also, $Y_i$s (conditioned on $x_i$) are not i.i.d, just independent. Looks like what you are trying to do finally is a likelihood ratio test, but remember the statistic $-2\ln \lambda$ ($\lambda$ being the likelihood ratio) is only asymptotically chi-square. $\endgroup$ – StubbornAtom Oct 13 '19 at 10:52
  • $\begingroup$ You can also get a test from a confidence interval but are you sure about the interval you got here? $\endgroup$ – StubbornAtom Oct 13 '19 at 11:18
  • 1
    $\begingroup$ I am pretty sure, but maybe the $z_{1-\alpha/2}$ needs to be $z_{\alpha/2}$... $\endgroup$ – hello Oct 13 '19 at 11:35
  • $\begingroup$ Also, I don't know what the distribution of the $x_i$s is. That Is not given. $\endgroup$ – hello Oct 13 '19 at 11:37
  • $\begingroup$ Then I suppose the data is just $(Y_1,\ldots,Y_n)$ instead of $(Y_1,x_1),\ldots,(Y_n,x_n)$ where the $Y_i$s are independent normal variables? Your notation made me think there is a joint distribution. $\endgroup$ – StubbornAtom Oct 13 '19 at 11:44

Assuming you observe $\mathbf Y=(Y_1,Y_2,\ldots,Y_n)$ where $Y_i\sim N(\theta x_i,1)$ independently for all $i$ and $x_i$ is fixed.

Indeed, MLE of $\theta$ is given by $$\hat\theta(\mathbf Y)=\frac{\sum_{i=1}^n x_i Y_i}{\sum_{i=1}^n x_i^2}$$

By the reproductive property of normal distribution, we have an exact distribution for the MLE:$$\hat\theta\sim N\left(\theta,\frac{1}{\sum_{i=1}^n x_i^2}\right)$$

In other words, $$\sqrt{\sum_{i=1}^n x_i^2}\left(\hat\theta-\theta\right)\sim N(0,1)$$

Using this pivot, a $100(1-\alpha)\%$ confidence interval for $\theta$ is $$I=\left[\hat\theta-\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}},\hat\theta+\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}}\right]$$

That is, $$P_{\theta}[\theta\in I]=1-\alpha\quad,\forall\,\theta$$

Or, $$P_{\theta}[\theta\in I^c]=\alpha\quad,\forall\,\theta$$

So for some $\theta_0$, $$P_{\theta_0}[\theta\in I^c]=\alpha$$

This gives the following critical region of a size $\alpha$ test for testing $H_0:\theta=\theta_0$ against $H_1:\theta\ne\theta_0$:

$$\left\{\mathbf Y:\hat\theta(\mathbf Y)<\theta_0-\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}}\quad\text{ or }\quad \hat\theta(\mathbf Y)>\theta_0+\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}}\right\}$$

Other tests can be derived of course but this gives you a test directly using the confidence interval $I$.

  • $\begingroup$ Thank you!!! I think I understand :) $\endgroup$ – hello Oct 13 '19 at 15:41
  • $\begingroup$ You could try and see what the likelihood ratio test gives you. For that, the rule is to reject $H_0$ if $\lambda<c$ for some constant $c$. I don't see any actual need to use the large sample approximation of $-2\ln \lambda$ under $H_0$. The statement '$(\hat{\theta} - \theta_0)^2\sum_{i = 1}^{n} x_i^2 \sim \chi^2_1$ under $H_0$' is true in any case. $\endgroup$ – StubbornAtom Oct 13 '19 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.