# Find the MLE of $N(\theta,\theta)$

Suppose $X_1,\ldots,X_n$ are iid $N(\theta,\theta)$, with $\theta\in(0,\infty)$. Find the MLE of $\theta$. I got $\frac{\partial logL(x|\theta)}{\partial \theta}=-\frac{n}{2}\frac{1}{\theta}+\frac{\sum_{i=1}^nx_{i}^2}{2}\frac{1}{\theta^2}-\frac{n}{2}$. Then, I let $\frac{\partial logL(x|\theta)}{\partial \theta}=0$. Then, I have $-\frac{n}{2}\theta^2-\frac{n}{2}\theta+\frac{1}{2}\sum_{i=1}^nx_{i}^2=0$. Then, I find the solution is weird. Am I wrong?

• Why the sums go to $2$? Aug 16, 2015 at 17:49
• @PaoloLeonetti. which part? Aug 16, 2015 at 17:53
• All your summations go from $1$ to $2$ :P Aug 16, 2015 at 18:02
• @PaoloLeonetti. Edited. Aug 16, 2015 at 18:05
• Are you sure that in $log L$ there are no logs? :P Aug 16, 2015 at 18:08

I see there is some mistake in your calculations..

Given $x_1,\ldots,x_n$ the log-likelihood is proportional to $$\ell\colon \mathbf{R}^+\to \mathbf{R}\colon \theta\mapsto -n\ln \theta-\sum_{i=1}^n\left(\frac{x_i}{\theta}-1\right)^2.$$ Since we have to maximize $-\ell$, it is enough to see that $$\ell^\prime(\theta) \propto \frac{n}{\theta}-\sum_{i=1}^n\frac{2x_i}{\theta^2}\left(\frac{x_i}{\theta}-1\right).$$ Then, if I am not wrong, the solution is $$\hat{\theta}=-\overline{x}+\sqrt{\overline{x}^2+2\overline{x^2}}.$$

• You need to check one thing: $\theta>0$. But $\bar{x}$ can be negative!! Aug 16, 2015 at 18:10
• Well, the square root is greater in absolute value.. Aug 16, 2015 at 18:12
• Sorry commented is a haste.
– Saty
Aug 16, 2015 at 21:42
• Couldn't verify it using R-simulation. Dec 16, 2019 at 9:18

### Derivation

I think, you are (were) on the right way. Let me continue your computations:

Suppose $$X_1,\ldots,X_n$$ are iid $$N(\theta,\theta)$$, with $$\theta\in(0,\infty)$$. Find the MLE of $$\theta$$.

$$\frac{\partial \log L(x|\theta)}{\partial \theta}=-\frac{n}{2}\frac{1} {\theta}+\frac{\sum_{i=1}^nx_{i}^2}{2}\frac{1}{\theta^2}-\frac{n}{2}$$

$$\frac{\partial \log L(x|\theta)}{\partial \theta}=0$$

$$-\frac{n}{2}\theta^2-\frac{n}{2}\theta+\frac{1}{2}\sum_{i=1}^nx_{i}^2=0$$

And multiply by 2 $$-n\theta^2-n\theta+\sum_{i=1}^nx_{i}^2=0$$

Now solve for $$\theta$$:

$$\theta_{1,2} = \frac{n \pm \sqrt{n^2 + 4n\sum_{i=1}^nx_{i}^2}}{-2n} = \frac{-1 \mp \sqrt{1 + 4\frac{1}{n}\sum_{i=1}^nx_{i}^2}}{2}$$

Since $$1 < \sqrt{1 + 4\frac{1}{n}\sum_{i=1}^nx_{i}^2}$$ we have only one solution for $$\theta > 0$$

$$\bar{\theta}^{MLE}_n = \frac{\sqrt{1 + 4\frac{1}{n}\sum_{i=1}^nx_{i}^2}-1}{2} = \frac{1}{2}\left(\sqrt{1 + 4 \overline{x^2}} - 1\right)$$

### Proof of consistency

Let's proof a.s. convergance to true $$\theta$$ using LLN and second moment of the normal distribution.

Accourding to LLN (https://en.wikipedia.org/wiki/Law_of_large_numbers)

$$\overline{x^2} = \frac{1}{n}\sum_{i=1}^nx_{i}^2 \underset{n\rightarrow\infty}{\overset{a.s.}{\longrightarrow}} \mathbb{E}[x_i^2]$$

Second moment of $$N(\mu, \sigma^2)$$ is $$\mathbb{E}[x_i^2] =\mu^2 + \sigma^2 = \theta^2 + \theta$$

$$\bar{\theta}^{MLE}_n = \frac{\sqrt{1 + 4\overline{x^2}}-1}{2} = \frac{\sqrt{1 + 4\theta^2 + 4\theta}-1}{2} = \frac{\sqrt{(1 + 2\theta)^2}-1}{2} = \theta$$

### Simulation with R

You can verify this with the following simple R-script (n=100000, $$\theta$$ = 0.25) :

theta <- 0.25
n <- 100000
z <- rnorm(n, mean = theta, sd = sqrt(theta))
theta_MLE = (1 + sqrt(1 + 4*sum(z^2)/n))/2 - 1

# Output
theta_MLE
theta - theta_MLE


Output:

 0.2515441
 -0.001544133