Understanding MLE of $N(\theta, \theta)$

The maximum likelihood estimator of $$N(\theta, \theta)$$ is

$$\bar{\theta}^{MLE}_n = \frac{1}{2}(\sqrt{1 + 4 \overline{x^2}} - 1)$$

for $$\overline{x^2}=\frac{1}{n}\sum_{i=1}^nx_{i}^2$$

(See https://math.stackexchange.com/a/3478232/735298 for derivation)

Now playing with this estimator I tried some simple $$x_i$$s, like,

• $$x_1=x_2=...=x_n=1$$, that gives me

$$\bar{\theta}^{MLE}_n = \frac{1}{2}(\sqrt{1 + 4\cdot 1} - 1) = \frac{1}{2}(\sqrt{5} - 1)=0.618...$$

I expected here to get 1, since intuitively the mean 1 with any variance would be the most probable one.

Also for $$x_1=x_2=...=x_n=0$$ we get expected $$\bar{\theta}^{MLE}_n=0$$.

My questions:

• What's wrong with my intuition described above?
• What is the explanation of the $$\bar{\theta}^{MLE}_n = 0.618$$ for all $$x_i=1$$?
• here is a question: What is the probability that you observe $n$ values of $1$ from your normal dist? Commented Dec 16, 2019 at 11:04
• Zero of course, but I don't see, how it should help me to gain intuition of getting something like 0.618... And why then it works for 0? Commented Dec 16, 2019 at 12:50
• "Working" doesn't mean anything about meaningfulness. Commented Dec 16, 2019 at 14:41
• Instead of the probability of actually seeing the sample, look at the likelihood function. Why is the likelihood function for this sample improving as you decrease $\theta$ from $\theta=1$? The idea is that a $N(1,1)$ sample would have a lot more variation than this, you would expect to see samples scattered all around $(-1,3)$ ($2\sigma$ variation) and some occasional stragglers a little further out. If you reduce the variance a little bit, then the sample is a bit further from the mean but the chance that there would be so little variation in the sample goes up.
– Ian
Commented Dec 16, 2019 at 14:48
• @Ian I am not so happy with seeing $N(0,0)$: I feel pain in my eyes :-) Commented Dec 16, 2019 at 14:51

What's wrong with my intuition described above? What is the explanation of the $$\bar{\theta}^{MLE}_n = 0.618$$ for all $$x_i=1$$?

The problem with the data samples $$x_1=x_2=...=x_n=1$$ is that "are they really generated by $$\mathcal{N}(\theta,\theta)$$ ?" I mean yes, the mean is $$1$$. But is their variance $$1$$ ? Especially when $$n$$ grows larger and larger !!? No. Roughly speaking, the sampling distribution model has a hard time believing the data.

In contrast, the data samples $$x_1=x_2=...=x_n=0$$ fit the model $$\mathcal{N}(\theta,\theta)$$ perfectly for $$\theta = 0$$, and thus you could explain the data using the sampling distribution. That said, you could use any software such as Python or MATLAB to generate pseudorandom numbers sampled from a normal distribution with equal mean and variance. For example, on MATLAB, the following generates $$10^4$$ samples from $$\mathcal{N}(1,1)$$:

x = 1 + randn(10000,1)
0.5*(sqrt(1 + 4*mean(x.^2)) - 1)


which gives an answer $$\sim 1$$

Note that parameter $$\theta$$ is not only the mean of the distribution but also the variance. If all samples are $$1$$, the sample mean is $$1$$ and the sample variance is $$0$$. How should estimator for $$\theta$$ react to such a difference? It gives out something in between.

For zero values: there is complete agreement - the sample mean is $$0$$, the sample variance is $$0$$. It would be strange if the estimate did not give zero.