1
$\begingroup$

The maximum likelihood estimator of $N(\theta, \theta)$ is

$$ \bar{\theta}^{MLE}_n = \frac{1}{2}(\sqrt{1 + 4 \overline{x^2}} - 1) $$

for $\overline{x^2}=\frac{1}{n}\sum_{i=1}^nx_{i}^2$

(See https://math.stackexchange.com/a/3478232/735298 for derivation)

Now playing with this estimator I tried some simple $x_i$s, like,

  • $x_1=x_2=...=x_n=1$, that gives me

$$ \bar{\theta}^{MLE}_n = \frac{1}{2}(\sqrt{1 + 4\cdot 1} - 1) = \frac{1}{2}(\sqrt{5} - 1)=0.618... $$

I expected here to get 1, since intuitively the mean 1 with any variance would be the most probable one.

Also for $x_1=x_2=...=x_n=0$ we get expected $\bar{\theta}^{MLE}_n=0$.

My questions:

  • What's wrong with my intuition described above?
  • What is the explanation of the $\bar{\theta}^{MLE}_n = 0.618$ for all $x_i=1$?
$\endgroup$
8
  • $\begingroup$ here is a question: What is the probability that you observe $n$ values of $1$ from your normal dist? $\endgroup$
    – Math-fun
    Commented Dec 16, 2019 at 11:04
  • $\begingroup$ Zero of course, but I don't see, how it should help me to gain intuition of getting something like 0.618... And why then it works for 0? $\endgroup$ Commented Dec 16, 2019 at 12:50
  • $\begingroup$ "Working" doesn't mean anything about meaningfulness. $\endgroup$
    – Math-fun
    Commented Dec 16, 2019 at 14:41
  • 1
    $\begingroup$ Instead of the probability of actually seeing the sample, look at the likelihood function. Why is the likelihood function for this sample improving as you decrease $\theta$ from $\theta=1$? The idea is that a $N(1,1)$ sample would have a lot more variation than this, you would expect to see samples scattered all around $(-1,3)$ ($2\sigma$ variation) and some occasional stragglers a little further out. If you reduce the variance a little bit, then the sample is a bit further from the mean but the chance that there would be so little variation in the sample goes up. $\endgroup$
    – Ian
    Commented Dec 16, 2019 at 14:48
  • 1
    $\begingroup$ @Ian I am not so happy with seeing $N(0,0)$: I feel pain in my eyes :-) $\endgroup$
    – Math-fun
    Commented Dec 16, 2019 at 14:51

2 Answers 2

1
$\begingroup$

What's wrong with my intuition described above? What is the explanation of the $\bar{\theta}^{MLE}_n = 0.618$ for all $x_i=1$?

The problem with the data samples $$x_1=x_2=...=x_n=1$$ is that "are they really generated by $\mathcal{N}(\theta,\theta)$ ?" I mean yes, the mean is $1$. But is their variance $1$ ? Especially when $n$ grows larger and larger !!? No. Roughly speaking, the sampling distribution model has a hard time believing the data.

In contrast, the data samples $$x_1=x_2=...=x_n=0$$ fit the model $\mathcal{N}(\theta,\theta)$ perfectly for $\theta = 0$, and thus you could explain the data using the sampling distribution. That said, you could use any software such as Python or MATLAB to generate pseudorandom numbers sampled from a normal distribution with equal mean and variance. For example, on MATLAB, the following generates $10^4$ samples from $\mathcal{N}(1,1)$:

x = 1 + randn(10000,1)
0.5*(sqrt(1 + 4*mean(x.^2)) - 1)

which gives an answer $\sim 1$

$\endgroup$
1
$\begingroup$

Note that parameter $\theta$ is not only the mean of the distribution but also the variance. If all samples are $1$, the sample mean is $1$ and the sample variance is $0$. How should estimator for $\theta$ react to such a difference? It gives out something in between.

For zero values: there is complete agreement - the sample mean is $0$, the sample variance is $0$. It would be strange if the estimate did not give zero.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .