Find the MLE of $N(\theta,\theta)$

Question

Suppose $X_1,\ldots,X_n$ are iid $N(\theta,\theta)$, with $\theta\in(0,\infty)$. Find the MLE of $\theta$. I got $\frac{\partial logL(x|\theta)}{\partial \theta}=-\frac{n}{2}\frac{1}{\theta}+\frac{\sum_{i=1}^nx_{i}^2}{2}\frac{1}{\theta^2}-\frac{n}{2}$. Then, I let $\frac{\partial logL(x|\theta)}{\partial \theta}=0$. Then, I have $-\frac{n}{2}\theta^2-\frac{n}{2}\theta+\frac{1}{2}\sum_{i=1}^nx_{i}^2=0$. Then, I find the solution is weird. Am I wrong?

Answer 1

I see there is some mistake in your calculations..

Given $x_1,\ldots,x_n$ the log-likelihood is proportional to $$ \ell\colon \mathbf{R}^+\to \mathbf{R}\colon \theta\mapsto -n\ln \theta-\sum_{i=1}^n\left(\frac{x_i}{\theta}-1\right)^2. $$ Since we have to maximize $-\ell$, it is enough to see that $$ \ell^\prime(\theta) \propto \frac{n}{\theta}-\sum_{i=1}^n\frac{2x_i}{\theta^2}\left(\frac{x_i}{\theta}-1\right). $$ Then, if I am not wrong, the solution is $$ \hat{\theta}=-\overline{x}+\sqrt{\overline{x}^2+2\overline{x^2}}. $$

Answer 2

Derivation

I think, you are (were) on the right way. Let me continue your computations:

Suppose $X_1,\ldots,X_n$ are iid $N(\theta,\theta)$, with $\theta\in(0,\infty)$. Find the MLE of $\theta$.

$$ \frac{\partial \log L(x|\theta)}{\partial \theta}=-\frac{n}{2}\frac{1} {\theta}+\frac{\sum_{i=1}^nx_{i}^2}{2}\frac{1}{\theta^2}-\frac{n}{2} $$

$$ \frac{\partial \log L(x|\theta)}{\partial \theta}=0 $$

$$ -\frac{n}{2}\theta^2-\frac{n}{2}\theta+\frac{1}{2}\sum_{i=1}^nx_{i}^2=0 $$

And multiply by 2 $$ -n\theta^2-n\theta+\sum_{i=1}^nx_{i}^2=0 $$

Now solve for $\theta$:

$$ \theta_{1,2} = \frac{n \pm \sqrt{n^2 + 4n\sum_{i=1}^nx_{i}^2}}{-2n} = \frac{-1 \mp \sqrt{1 + 4\frac{1}{n}\sum_{i=1}^nx_{i}^2}}{2} $$

Since $1 < \sqrt{1 + 4\frac{1}{n}\sum_{i=1}^nx_{i}^2}$ we have only one solution for $\theta > 0$

$$ \bar{\theta}^{MLE}_n = \frac{\sqrt{1 + 4\frac{1}{n}\sum_{i=1}^nx_{i}^2}-1}{2} = \frac{1}{2}\left(\sqrt{1 + 4 \overline{x^2}} - 1\right) $$

Proof of consistency

Let's proof a.s. convergance to true $\theta$ using LLN and second moment of the normal distribution.

Accourding to LLN (https://en.wikipedia.org/wiki/Law_of_large_numbers)

$$ \overline{x^2} = \frac{1}{n}\sum_{i=1}^nx_{i}^2 \underset{n\rightarrow\infty}{\overset{a.s.}{\longrightarrow}} \mathbb{E}[x_i^2] $$

Second moment of $N(\mu, \sigma^2)$ is $\mathbb{E}[x_i^2] =\mu^2 + \sigma^2 = \theta^2 + \theta$

$$ \bar{\theta}^{MLE}_n = \frac{\sqrt{1 + 4\overline{x^2}}-1}{2} = \frac{\sqrt{1 + 4\theta^2 + 4\theta}-1}{2} = \frac{\sqrt{(1 + 2\theta)^2}-1}{2} = \theta $$

Simulation with R

You can verify this with the following simple R-script (n=100000, $\theta$ = 0.25) :

theta <- 0.25
n <- 100000
z <- rnorm(n, mean = theta, sd = sqrt(theta))
theta_MLE = (1 + sqrt(1 + 4*sum(z^2)/n))/2 - 1

# Output
theta_MLE
theta - theta_MLE

Output:

[1] 0.2515441
[1] -0.001544133