0
$\begingroup$

tl;dr: What is wrong with this MLE estimator? Does it not satisfy the conditions for consistency or did I make a mistake in the calculation.

I am trying to compute the MLE estimator for parameter $\theta > 0$ for sequence of observed i.i.d random variables $x_1, ..., x_n$ generated from a gaussian distribution $\mathcal{N}(\theta, \theta)$.
I went through the following calculations to maximize the log-likelihood:

$$ log(p(x_1, ..., x_n|\theta)) = \sum log\frac{1}{\sqrt{2\pi\theta}}exp(-\frac{(x_i-\theta)^2}{2\theta}) \; \alpha \; -\frac{n}{2}log(\theta) - \sum\frac{(x_i-\theta)^2}{2\theta} $$

Where I omitted the term containing $\pi$ in the last derivation as it disappears in the derivative calculated below:

$$ \frac{d log(p(x_1, ..., x_n|\theta))}{d\theta} = -\frac{n}{2\theta} + \sum\frac{2(x_i - \theta)}{2\theta} + \frac{\sum(x_i - \theta)^2}{2\theta^2} = \frac{-n\theta + \theta \sum2(x_i - \theta) + \sum(x_i - \theta)^2}{2\theta^2} = \frac{\sum x_i^2 - n\theta^2 - n\theta}{2\theta^2} $$

Setting the numerator to zero to find the maximum gives me:

$$ \frac{n +\sqrt{n^2 + 4n\sum x_i^2}}{-2n} $$

Where we don't consider the other root of the equation as it is negative.
Now, this is a question from an MIT OCW assignment question and I am asked to also prove the consistency of this estimator. Hence we have:

$$ lim_{n->\infty}E\frac{n +\sqrt{n^2 + 4n\sum x_i^2}}{-2n} = lim_{n->\infty}E\sqrt{\frac{1}{4} + \frac{\sum x_i^2}{n}} - \frac{1}{2} $$

Now I am pretty sure that this value does not equal $\theta$ and hence the estimator not consistent. — One possibility might be that I made a mistake somewhere, but I have checked it a couple of times. So I went and checked if the log-likelihood was in fact concave or not to see whether it really could be maximized by finding the point with the derivative equalling zero. I got this expression for the second derivative of the log-likelihood:

$$ \alpha\; \; \frac{n\theta - 2\sum x_i^2}{\theta^3} $$

When we know $\theta$ to be positive (which we do) the expectation of this is in fact negative. But does this mean we can in fact use convex optimization and find the maximum (minimum) by setting the derivative equal to zero?

The side question:
2. Is the convex optimization valid given the aforementioned argument?

The answer to the first part has been provided by another user below. I also found out the answer to this second part: It basically comes down to the actual $x_i$s you have in each instance - as the maximization is taking place over $\theta$ and if the second derivative is negative for a specific instance of data, i.e: $x_i$s, then minimization over $\theta$ can be done by setting the first derivative to zero.

$\endgroup$
  • $\begingroup$ The estimator $\hat\theta_n$ is consistent for $\theta$ if $\hat\theta_n \xrightarrow{p}\theta$ as $n\to\infty$. There is no expectations. $\endgroup$ – NCh Feb 24 at 2:14
2
$\begingroup$

The MLE is $$ \hat{\theta}_n=\sqrt{\frac{1}{4}+\frac{1}{n}\sum_{i=1}^n x_i^2}-\frac{1}{2} $$ By the WLLN, $$ \hat{\theta}_n\xrightarrow{p}\sqrt{\frac{1}{4}+\mathsf{E}x_1^2}-\frac{1}{2}=\sqrt{(1/2+\theta)^2}-\frac{1}{2}=\theta. $$

$\endgroup$
  • $\begingroup$ How did you get the expectation inside the square root? I don't see how WLLN could do that. Though I can see the intuition behind it. $\endgroup$ – user2268997 Feb 24 at 1:26
  • 1
    $\begingroup$ Continuous mapping theorem and the WLLN: $n^{-1}\sum x_i^2\xrightarrow{p} \mathsf{E} x_1^2=\theta+\theta^2$. $\endgroup$ – d.k.o. Feb 24 at 2:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.