2
$\begingroup$

Let $X_1,...,X_n\stackrel{iid}{\sim}\text{Normal}(\theta,1)$ and define\begin{align} Y_i=\begin{cases}1&X_i>0\\ 0&X_i\leq 0. \end{cases} \end{align} Let $\psi=\mathbb{P}(Y_1=1)$ and $\widehat{\psi}$ be the maximum likelihood estimator (MLE) of $\psi$.

I want to show that if the data are not normal, then the MLE $\widehat{\psi}$ is not consistent and show to which it converges, if it converges at all.

I determined the MLE, which is given by $\widehat{\psi}=\Phi(\overline{X})$, where $\Phi$ is the CDF of the standard normal distribution. I have looked at the proof of consistency of the MLE but so far I haven't found a direct relation with the fact that the data are normal. Any ideas?

$\endgroup$
  • $\begingroup$ Surely the proof that an MLE is consistent uses the fact that the data comes from the distribution you assumed when you derived the MLE. $\endgroup$ – spaceisdarkgreen Oct 31 '17 at 15:59
2
$\begingroup$

To begin, suppose that the normality assumption holds. You are asked to find the MLE for $\psi \equiv \mathbb{P}(X_i >0)$ where $X_i\sim N(\theta,1)$. By definition: $$\psi = \mathbb{P}(X_i > 0) = \mathbb{P}(Z + \theta > 0) = \mathbb{P}(Z > - \theta) = 1 - \mathbb{P}(Z \leq -\theta) = 1 - \Phi(-\theta) = \Phi(\theta)$$

where $Z\sim N(0,1)$. The MLE for $\theta$ is clearly $\bar{X}_n$ so by the invariance of ML estimators $\Phi(\bar{X}_n)$ is a the MLE for $\psi = \Phi(\theta)$.

Now suppose that $X_1, X_2, \cdots, X_n \sim \mbox{ iid } F$ where $F$ is some distribution function with mean $\theta$ and finite variance that might not be $N(\theta, 1)$. As before we desire an estimator for $\psi \equiv \mathbb{P}(X_i > 0)$. By definition $\psi = 1 - F(0)$. By the weak law of large numbers, $\bar{X}_n$ converges in probability to $\theta$ regardless of whether $F$ is a $N(\theta, 1)$ distribution. And by the continuous mapping theorem it follows that $\Phi(\bar{X}_n)$ converges in probability to $\Phi(\theta)$.

Now the question is simply whether $\Phi(\theta)$ equals $1 - F(0)$. If $F$ is not a $N(\theta,1)$ distribution, this equality will not in general hold. In particular examples, however it could still hold even if $F$ is non-normal. For example, suppose that $\theta = 0$ so that $\psi = \Phi(0) = 0.5$. Now suppose that $F$ is the CDF of a random variable that is equally likely to take on the values $-1$ and $1$ and never takes on any other values. Then $F$ has mean $0$, which equals $\theta$, and $1 - F(0) = 0.5$, which equals $\psi$.

In contrast to the MLE, the method of moments estimator $\widetilde{\psi} = n^{-1} \sum_{i=1}^n \mathbf{1}(X_i > 0)$ remains consistent for $\psi$ under very general conditions, regardless of whether $F$ is a $N(\theta, 1)$ distribution.

$\endgroup$
2
$\begingroup$

This question is somewhat awkwardly posed. I will assume that $X_i$ are i.i.d. from a distribution family parametrized by a location parameter $\theta$ which is not the normal distribution family. I will assume $X$ is integrable and (without additional loss of generality) that its median is $a+\theta$ and mean is $E(X) = b + \theta.$

We have $\hat \psi \to \Phi(b+\theta)$ a.s. So in order to have consistency we need $$P(X \ge 0)= \Phi(b+\theta)$$ for all $\theta.$

We have $$P(X \ge 0)=1- F(-(a+\theta))$$ where $F$ is the CDF of $X$ so consistency would imply that $F(-(a+\theta)) = \Phi(-(b+\theta))$ for all $\theta$ so that $F$ is a (possibly shifted) normal distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.