1
$\begingroup$

Let $X_1, X_2, ..., X_n$ be iid. random variables with pdf. exponential. Let $H_0 = \theta = \theta_0 $ and $H_1 = \theta = \theta_1$. Then test statistic for likelihood ratio would be:

$$ LR = \frac{f(X_1, X_2, ..., X_n \mid\theta_0)}{f(X_1, X_2, ..., X_n \mid\theta_1)} = \frac{\theta_0^{-n}\exp \{-\sum_i X_i/\theta_0 \}}{\theta_1^{-n}\exp \left\{-\sum_i X_i/\theta_1 \right\}} = \left(\frac{\theta_0}{\theta_1}\right)^{-n}e^{\left(\frac{1}{\theta_1} - \frac{1}{\theta_0}\right) \sum_{i} X_{i}}$$

How to show that $LR \rightarrow \chi^2_1 \ $ ? Thanks in advance!

$\endgroup$
2
  • $\begingroup$ I am thinking $H_0:\theta = \theta_0$ and $H_1:\theta \neq \theta_0.$ In this case, you need to set $\theta_0^* = \theta_0$ and $\theta_1^* = \bar{x}_n.$ See the theorem below, cheers. $\endgroup$
    – William M.
    Commented Oct 19, 2021 at 18:53
  • $\begingroup$ By the way, as stated, $LR$ will not converge to $\chi^2,$ it is $-2 \log LR$ that converges. $\endgroup$
    – William M.
    Commented Oct 19, 2021 at 22:29

1 Answer 1

1
$\begingroup$

Let $x \mapsto f(x \mid \theta)$ be a parametrised density function on a sample space $\mathrm{X}$ (so that $x$ are the possible observations) and with parameter $\theta$ assumed a priori to lie in some $\Omega_0 \subset \mathbf{R}^{d_0}.$ (Do not confuse this with a priori Bayesian distributions.) Suppose that $n$ independent samples are taken and write $$ f_n(\mathbf{x}_n \mid \theta) = f_n(x_1, \ldots, x_n \mid \theta) = \prod_{k = 1}^n f(x_k \mid \theta) $$ for the likelihood of the sample.

Suppose $\Omega_1$ is an open set such that $\Omega_0 \subset \Omega_1$ and we further assume that $\Omega_0$ is embedded in a $d_0$-dimensional subspace of $\mathbf{R}^{d_1}$ ($d_0 = 0$ not excluded, meaning $\Omega_0$ is a single point). Let $$ \ell_n^*(\mathbf{x}_n \mid \mathrm{H}_i) = \sup_{\theta \in \Omega_i} \log f_n(\mathbf{x}_n \mid \theta) $$ the log-likelihood optimised over the hypothesis $\mathrm{H}_i$ that $\theta \in \Omega_i.$

Wilk's theorem: Suppose that $f$ satisfy some measure-theoretic regularity conditions, and that for each $n,$ $\theta \mapsto f_n(\mathbf{x}_n \mid \theta)$ has maximimisers $$ \theta_1^* = \theta_{1,n}^*(\mathbf{x}_n) $$ in the interior of $\Omega_1,$ and $$ \theta_0^* = \theta_{0,n}^*(\mathbf{x}_n) $$ the interior of $\Omega_0$ relative to the subspace it is embbed into. In other words, we assume that $$ \ell_n^*(\mathbf{x_n} \mid \mathrm{H}_i) = \log f_n(\mathbf{x}_n \mid \theta_i^*). $$ Then, as $n \to \infty$ ("as the number of sampled observations increases unlimitedly") $$ -2\log \dfrac{f_n(\mathbf{x}_n \mid \theta_0^*)}{f_n(\mathbf{x}_n \mid \theta_1^*)} = 2 \big( \ell_n^*(\mathbf{x}_n \mid \mathrm{H}_1) - \ell_n^*(\mathbf{x}_n \mid \mathrm{H}_0) \big) \to \chi^2_{d_1 - d_0} $$

Notes:

  1. Wilk's theorem uses the ratio of likelihoods principle. The likelihood of a sample is a metric that, well, estimates how likely is to see what we saw if the parameter we assume is $\theta.$ The ratio of likelihoods (likelihood ratio) $\dfrac{f_n(\mathbf{x}_n \mid \theta_0^*)}{f_n(\mathbf{x}_n \mid \theta_1^*)}$ is always $\leq 1,$ so the log-likelihood ratio is therefore $\geq 0.$ The likelihood ratio principle state that if the likelihood ratio is too small (equiv. the log-likelihood is too big) then the hypothesis $\mathrm{H}_0$ is incompatible with the data and as such there is evidence suggesting that $\mathrm{H}_1$ holds. How unlikely depends on the actual density $f,$ however, Wilk's theorem states that for large samples, we do not depend (thankfully) on $f$ but on the $\chi^2$-disitribution (so, Wilk's theorem is sort of a central limit theorem for the likelihood ratio principle).

  2. Wilk's theorem can be extended to the case where $\Omega_0$ is a single point (i.e. specifies the parameter exactly).

  3. I am not sure if Wilk's theorem has an extension (or what would that extension even mean) if you specify both parameters. The usual proof uses that $\Omega_0 \subset \Omega_1$.

  4. To use Wilk's theorem, you need to find the ratio of the likelihoods evaluated at the optimisers of both hypotheses.

  5. The degrees of freedom will then be the difference in dimensions of the two hypothesis, namely $d_1 - d_0.$ When you specify both hypothesis to be a single parameter, you will reach a zero, so certainly something is wrong with your formulation.

  6. Check the following link too https://stats.stackexchange.com/questions/101520/what-are-the-regularity-conditions-for-likelihood-ratio-test

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .