Given that $X_{1},...,X_{n}$ are i.i.d random variables with joint distribution $f(x\mid \theta) $ with 1 dimensional parameter $\theta$ and let $\hat\theta$ be the maximum likelihood estimator of $\theta$.

Based on the Wilks theorem,under null hypothesis that $H_{0}: \theta=\theta_{0}$ ,one have that $-2\log\left(f(x\mid \theta_{0})/f(x\mid \hat\theta)\right)\to \chi_{1}^{2}$ as $n\to\infty$.

Since we know that $E\chi_{1}^{2}=1$, is there anyway I can find the value of the following integral (expectation of log likelihood ratio) or at least asymptotics

$$-2\int f(x\mid \theta_{0})\log\left(\frac{f(x\mid \theta_{0})}{f(x\mid \hat\theta)}\right)\,dx=~?$$

My guess is that the integral should be around 1. Anyone please share some idea or references with the above integral.

  • $\begingroup$ the integral is the KL divergence, which is always positive, so the whole term should be negative $\endgroup$ – Seyhmus Güngören Mar 2 '16 at 23:49
  • $\begingroup$ @SeyhmusGüngören, it is positive since the denominator in the log is maximum likelihood. $\endgroup$ – lzstat Mar 3 '16 at 15:54
  • $\begingroup$ Isn't it another density with another parameter? So the maximum likelihood will go to the true parameter as n goes to infinity. As a result there will be another density. $\endgroup$ – Seyhmus Güngören Mar 3 '16 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.