6
$\begingroup$

I came across the term second Bartlett identity in the wikipedia link: https://en.wikipedia.org/wiki/Variance_function However could not find detail about it. Can anyone help me to understand what this identity is about. More importantly I want to know under what assumption this identity is true.

$\endgroup$
0
8
$\begingroup$

The Bartlett identities are identities related to expectations of derivatives of the log likelihood function. I will stay in the setting where a random variable $X$ has a non-zero density $f$ that depends on one parameter $\theta$, but these ideas easily extend to the setting where you have a parameter vector. The Bartlett identities now arise when considering the expectation of derivatives of $\ell(\theta;x) = \log(f(x;\theta) )$ with respect to $\theta$. For every $\theta$ we have $$ \int_{\mathbb{R}} f(x;\theta) dx = 1. $$ Assuming (!) that you can interchange partial derivatives and integrals you now get by differentiating both sides $$ \frac{\partial}{\partial \theta} \int_{\mathbb{R}} f(x;\theta) dx = \int_{\mathbb{R}}\frac{\partial}{\partial \theta} f(x;\theta) dx = 0. $$ We can slightly rewrite the term in the middle to relate it to the expectation of the first derivative of $\ell$ as follows $$ \int_{\mathbb{R}}\frac{\partial}{\partial \theta} f(x;\theta) dx = \int_{\mathbb{R}}\frac{\frac{\partial}{\partial \theta} f(x;\theta)}{f(x;\theta)} f(x;\theta) dx = \mathbb{E}\left[\frac{\partial}{\partial \theta} \log(f(X;\theta))\right] $$ leading to $$ \mathbb{E}\left[\frac{\partial}{\partial \theta} \ell(\theta;X)\right]= 0. $$ This is your first Bartlett identity. But you don't have to stop there. Consider the second derivative $$ \frac{\partial^2}{\partial \theta^2} \ell(\theta;x) = \frac{f''(x;\theta)}{f(x;\theta)} - \left( \frac{f'(x;\theta)}{f(x;\theta)}\right)^2 $$ With the same approach as above (again interchanging derivatives and integrals) and taking a second derivative now yields $$ \int_{\mathbb{R}} \frac{\partial^2}{\partial \theta^2} f(x;\theta)dx = 0, $$ or using the same trick as above $$ \mathbb{E}\left[\frac{f''(X;\theta)}{f(X;\theta)}\right] = 0. $$ Using the second derivative of $\ell$ we thus have $$ \mathbb{E}\left[\frac{\partial^2}{\partial \theta^2} \ell(\theta;X)\right] + \mathbb{E}\left[\left(\frac{\partial}{\partial \theta} \ell(\theta;X)\right)^2\right] = 0. $$ Given that $\mathbb{E}\left[\frac{\partial}{\partial \theta} \ell(\theta;X)\right]= 0$ the second term can be identified as the variance of $\frac{\partial}{\partial \theta} \ell(\theta;X)$. This leads to the formula encountered on wikipedia $$ \mathbb{E}\left[\frac{\partial^2}{\partial \theta^2} \ell(\theta;X)\right] + \mbox{Var}\left(\frac{\partial}{\partial \theta} \ell(\theta;X)\right) = 0. $$ and is referred to as the second Bartlett identity. You don't have to stop there either. In fact (infinitely) more relations can be obtained, see Equation (1) in the original paper: M. S. Bartlett. Approximate Confidence Intervals. Biometrika, Vol. 40, No. 1/2 (Jun., 1953), pp. 12-19. The Bartlett identities are further related to maximum likelihood theory (Cramer-Rao lower bound, Fisher information) and can be used to construct or improve statistical tests (see for example: Cordeiro, Gauss M., Cribari-Neto, Francisco: An Introduction to Bartlett Correction and Bias Reduction. 2014.).

$\endgroup$
7
  • $\begingroup$ Thank you, 1+ this identity does not hold sometimes as mentioned in www3.stat.sinica.edu.tw/sstest/oldpdf/A21n11.pdf at section 2.2 just before equation 2.4. Any comments would be appreciated. Is it because integral of the likelihood is not 1? $\endgroup$
    – Creator
    Nov 23 '16 at 19:18
  • $\begingroup$ As far as I understand you don't get the Fisher information in these models because the likelihood is misspecified in the sense that you don't consider the full likelihood. Maybe the lecture notes on Unbiased Estimating Equations can serve as a starting point: stat.umn.edu/geyer/8112/notes/equations.pdf . Godambes original paper is: An Optimum Property of Regular Maximum Likelihood Estimation. V. P. Godambe. The Annals of Mathematical Statistics, Vol. 31, No. 4 (Dec., 1960), pp. 1208-1211. $\endgroup$ Nov 24 '16 at 15:02
  • $\begingroup$ I want to assume that as likelihood is insufficient/misspecified/incomplete so the first integral of your answer is not 1 and hence Barlett identity's are not true. Would you please confirm if it is correct? $\endgroup$
    – Creator
    Nov 24 '16 at 19:05
  • $\begingroup$ The first integral is always 1 - simply by virtue of $f$ being a density function. "Full likelihood" in my previous comment does not refer to an integration domain smaller than $\mathbb{R}$. Rather I mean that you take the "full" joint density $f_{X_1,\ldots,X_n}$ into account when constructing the maximum likelihood estimator. If you instead use another function in place of $f_{X_1,\ldots,X_n}$ in constructing your estimator you get the effect described in the papers. $\endgroup$ Nov 24 '16 at 19:18
  • $\begingroup$ I was wondering about that ; Then why the reference (www3.stat.sinica.edu.tw/sstest/oldpdf/A21n11.pdf) says Barlett identity is not valid. I see the first integral as the only required assumption for Barlett identities. Any comments please. $\endgroup$
    – Creator
    Nov 24 '16 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.