# Consistency of maximum likelihood estimator with non-normal data

Let $X_1,...,X_n\stackrel{iid}{\sim}\text{Normal}(\theta,1)$ and define\begin{align} Y_i=\begin{cases}1&X_i>0\\ 0&X_i\leq 0. \end{cases} \end{align} Let $\psi=\mathbb{P}(Y_1=1)$ and $\widehat{\psi}$ be the maximum likelihood estimator (MLE) of $\psi$.

I want to show that if the data are not normal, then the MLE $\widehat{\psi}$ is not consistent and show to which it converges, if it converges at all.

I determined the MLE, which is given by $\widehat{\psi}=\Phi(\overline{X})$, where $\Phi$ is the CDF of the standard normal distribution. I have looked at the proof of consistency of the MLE but so far I haven't found a direct relation with the fact that the data are normal. Any ideas?

• Surely the proof that an MLE is consistent uses the fact that the data comes from the distribution you assumed when you derived the MLE. – spaceisdarkgreen Oct 31 '17 at 15:59

To begin, suppose that the normality assumption holds. You are asked to find the MLE for $\psi \equiv \mathbb{P}(X_i >0)$ where $X_i\sim N(\theta,1)$. By definition: $$\psi = \mathbb{P}(X_i > 0) = \mathbb{P}(Z + \theta > 0) = \mathbb{P}(Z > - \theta) = 1 - \mathbb{P}(Z \leq -\theta) = 1 - \Phi(-\theta) = \Phi(\theta)$$

where $Z\sim N(0,1)$. The MLE for $\theta$ is clearly $\bar{X}_n$ so by the invariance of ML estimators $\Phi(\bar{X}_n)$ is a the MLE for $\psi = \Phi(\theta)$.

Now suppose that $X_1, X_2, \cdots, X_n \sim \mbox{ iid } F$ where $F$ is some distribution function with mean $\theta$ and finite variance that might not be $N(\theta, 1)$. As before we desire an estimator for $\psi \equiv \mathbb{P}(X_i > 0)$. By definition $\psi = 1 - F(0)$. By the weak law of large numbers, $\bar{X}_n$ converges in probability to $\theta$ regardless of whether $F$ is a $N(\theta, 1)$ distribution. And by the continuous mapping theorem it follows that $\Phi(\bar{X}_n)$ converges in probability to $\Phi(\theta)$.

Now the question is simply whether $\Phi(\theta)$ equals $1 - F(0)$. If $F$ is not a $N(\theta,1)$ distribution, this equality will not in general hold. In particular examples, however it could still hold even if $F$ is non-normal. For example, suppose that $\theta = 0$ so that $\psi = \Phi(0) = 0.5$. Now suppose that $F$ is the CDF of a random variable that is equally likely to take on the values $-1$ and $1$ and never takes on any other values. Then $F$ has mean $0$, which equals $\theta$, and $1 - F(0) = 0.5$, which equals $\psi$.

In contrast to the MLE, the method of moments estimator $\widetilde{\psi} = n^{-1} \sum_{i=1}^n \mathbf{1}(X_i > 0)$ remains consistent for $\psi$ under very general conditions, regardless of whether $F$ is a $N(\theta, 1)$ distribution.

This question is somewhat awkwardly posed. I will assume that $X_i$ are i.i.d. from a distribution family parametrized by a location parameter $\theta$ which is not the normal distribution family. I will assume $X$ is integrable and (without additional loss of generality) that its median is $a+\theta$ and mean is $E(X) = b + \theta.$

We have $\hat \psi \to \Phi(b+\theta)$ a.s. So in order to have consistency we need $$P(X \ge 0)= \Phi(b+\theta)$$ for all $\theta.$

We have $$P(X \ge 0)=1- F(-(a+\theta))$$ where $F$ is the CDF of $X$ so consistency would imply that $F(-(a+\theta)) = \Phi(-(b+\theta))$ for all $\theta$ so that $F$ is a (possibly shifted) normal distribution.