5
$\begingroup$

Suppose I have a sequence of densities

$$ f_{X_n}(x) = \exp[\ell_n(x)], \qquad (x \in A). $$

My goal is to prove a statement like $\sqrt n (X_n - \mu) \to N(0, \sigma^2)$ in distribution, for an appropriate $\sigma^2$ and some choice of $\mu$. We often teach undergraduates to do this using moment generating functions or characteristic functions, or by direct manipulation of the cdf.

By contrast, I would like to do this by expanding $f_{X_n}(x)$ about the mode $\hat x_n$ (suppose $f_{X_n}(x)$ is unimodal with $\ell''_n(x) < 0)$. Then we expand $T_n = \sqrt n (X_n - \hat x_n)$

$$ f_{T_n}(t) \approx n^{-1/2}f_{X_n}(\hat x_n) \exp \left\{\frac 1 {2n} t^2 \ell''_n(\hat x_n)\right\}. $$

Under the condition that $\hat x_n = \mu + o(n^{-1/2})$ and $\ell''_n(\mu) / n \to -I(\mu)$ for some function $I(\mu)$, one then might guess

$$ T_n \stackrel{\cdot}{\sim} N(0, I(\mu)^{-1}). $$

Now for the question: I'm essentially looking for conditions which allow me to make this result rigorous. The heuristic seems very similar to what one does when trying to prove asymptotic normality of maximum likelihood estimators, and so I would hope for some conditions in terms of $\ell''_n(x)$ and $I(x)$, or perhaps a third derivative $\ell'''_n(x)$.

The best I can do at the moment is try to expand things out $$ f_{T_n}(t) = n^{-1/2} \exp\left[\ell_n (\hat x_n) + \underbrace{t / \sqrt n \ell_n(\hat x_n)}_{=0} + \frac{t^2}{2n}\ell''_n(\hat x_n + n^{-1/2} t^\star)\right] $$ with remainder $\frac 1 {2n} t^2 \ell''(\hat x_n + n^{-1/2} t^\star)$, which works for showing that the unnormalized density converges to a Gaussian density. But this doesn't finish the job because I still have to show that the normalizing constant converges to the right answer if I want to finish off the proof with Scheffe's theorem. I could try to finish it by applying dominated convergence to the exponential term, but this hasn't worked so far for the problem I'm interested in, and doesn't feel like a sensible thing to do anyways.

$\endgroup$
2
+150
$\begingroup$

See "Testing statistical hypotheses" by E.L. Lehmann (196), Chap. 6 regarding regularity conditions. If you don't have access to that, here is a link to them. See p. 6-12 of the pdf.

To summarize, you need to demonstrate a bit more than just consistency of the mode. Specifically, you need to show:

  1. $\lim_{n \to\infty} E(X_n)=\lim_{n \to\infty} \int xf_n(x) dx = \mu < \infty$
  2. $\lim_{n \to\infty} Var(X_n) = I(\mu)^{-1}\in[0,\infty)$
  3. The quadratic approximation needs to be unfiormly $O_p\left(\frac{1}{\sqrt{n}}\right)$ in a neighborhood around $\mu$.
  4. $\mu \notin \{\inf A,\sup A\}$

I think showing the above plus whatever arguments strike you from the link I sent you should do the trick. Also, to assuage your worries about the normalizing constant, see this paper by Geyer. He demonstrates that as long as the log-likehood approaches a quadratic, then the underlying variable is asymptotically normal...no CLT or LLN needed. It's based on Le Cam's local mixed normality and direct analysis of the log-likehood function.

$\endgroup$
  • $\begingroup$ Thanks, I will check out the regularity conditions that you linked. But I may be missing something to your closing paragraph as, it seems to me, if the normalizing constants happened to converge then the problem would be done because I would have pointwise convergence of the (normalized) density to the normal pdf, which suffices to prove convergence in total variation in view of Scheffe's lemma. $\endgroup$ – guy Jan 26 '15 at 22:12
  • $\begingroup$ It's also worth mentioning that, while it does seem similar to (and I wouldn't be surprised if it is essentially the same as) the maximum likelihood or the $M$-estimation problem, I'm not directly minimizing an objective function to get the random variable I'm studying the behavior of. The classical regularity conditions focus on $M$-estimators, so to use them I would either need to turn my random variable into an $M$-estimator, or figure out why the regularity conditions also apply to my RV. $\endgroup$ – guy Jan 26 '15 at 22:19
  • $\begingroup$ @guy note that by using the unique mode of your densities, you are effectively maximizing the likelihood. It just so happens that your mode converges to some parameter $\mu$ in probability. The first two of my conditions are needed to ensure that you can interpret the modes of the densities as a sequence of estimators of $\mu$ that converge in expected value (and probability) to the true parameter value $\mu$. $\endgroup$ – user76844 Jan 27 '15 at 3:47
  • $\begingroup$ @guy In essence, you can lean on MLE theory for this problem because one can construct a sequence of observations where the density of the "MLE" (read, mode) for $\mu$, after observing the $n^{th}$ value, has the density $f_n$ (sampling may or may not be iid, but the sequene of functions can only depend on $n$), then your problem is equivalent to MLE of $\mu$ under a particular model. $\endgroup$ – user76844 Jan 27 '15 at 3:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.