1
$\begingroup$

Suppose $X_1, \dots, X_n$ are iid from a uniform distribution in $[-a, a]$. Their PDF is thus given by $$f(x) = \dfrac{1}{2a}\mathbb{I}(x \in [-a ,a])\text{.}$$ $\mathbb{I}$, above, denotes the indicator function. I wish to find a MLE for $a$.

The likelihood function is given by $$L(a \mid x_1, \dots, x_n) = \left(\dfrac{1}{2a}\right)^n\prod_{i=1}^{n}\mathbb{I}(x_i \in [-a, a])\text{.}$$ I am quite certain that one cannot do the traditional method of finding the loglikelihood, setting its derivative equal to $0$, and solving for $a$. I know there's some trick to doing this - perhaps involving an order statistic - but it has been a long time since I've seen the trick. Can I have a hint, not a complete solution on how to proceed with this problem?

Edit: Intuitively speaking, I think that if I wanted to choose $a$ to maximize $L$, I wouldn't choose $a$ to be a negative value. Actually, I would choose $a$ to be the smallest value of $x_i$ which is greater than $0$ - since it will be placed in the denominator of $L$ and as $x_i$ is larger, $L$ gets smaller.

$\endgroup$
2
$\begingroup$

You still are maximizing the likelihood. In this case you can do it by inspection. The likelihood is zero if $|x_i|>a$ for any $i$. On the other hand, the product is just $1$ if $|x_i| \leq a$ for all $i$, so once that is the case, the likelihood function is $(2a)^{-n}$, which is a decreasing positive function of $a$. So the maximum occurs when...?

$\endgroup$
  • $\begingroup$ See my edit. Is my hunch correct? $\endgroup$ – Clarinetist May 18 '16 at 13:05
  • $\begingroup$ @Clarinetist Not quite, what happens if your data is $\{ -10,1 \}$? $\endgroup$ – Ian May 18 '16 at 13:06
  • $\begingroup$ Hmm, still not quite seeing it. Would it be $\min_{i}|x_i|$? If I were to use the idea I had above, $L(\hat{a} = 1 \mid -10, 1)$ would be $\dfrac{1}{2^2} = \dfrac{1}{4}$. I'm not sure if I can maximize it any more than this. $\endgroup$ – Clarinetist May 18 '16 at 13:13
  • $\begingroup$ I would think the maximum would occur around $0$. $\endgroup$ – Clarinetist May 18 '16 at 13:14
  • $\begingroup$ @Clarinetist No, $L(1 | -10,1)$ is $0$ because $-10$ is not in $[-1,1]$. Then all of a sudden $L(10 | -10,1)$ is $\frac{1}{(2 \cdot 10)^2}$... $\endgroup$ – Ian May 18 '16 at 13:15
3
$\begingroup$

If there exists a $x_i\notin [-a,a]$, we have $L=0$. If all $x_i\in [-a,a]$, we have $L\left( {\left. a \right|{x_1},...,{x_n}} \right)=(1/2a)^n$, or $L\left( {\left. a \right|{x_1},...,{x_n}} \right)=(1/2a)^n$ when $0 \leqslant \mathop {\max }\limits_i \left| {{x_i}} \right| \leqslant a$. Then, $$\log L\left( {\left. a \right|{x_1},...,{x_n}} \right) = \left\{ \begin{gathered} n\log \left( {\frac{1} {{2a}}} \right),\,\,\,0 \leqslant \mathop {\max }\limits_i \left| {{x_i}} \right| \leqslant a, \hfill \\ 0,\,\,\,\,\,\,\,otherwi\operatorname{se} \\ \end{gathered} \right.$$ This gives $$\frac{{\partial \log L}}{{\partial a}} = \left\{ \begin{gathered} - n/a,\,\,\,0 \leqslant \mathop {\max }\limits_i \left| {{x_i}} \right| \leqslant a, \\ 0,\,\,\,\,\,\,\,otherwi\operatorname{se} \\ \end{gathered} \right.$$ Clearly, $\frac{{\partial \log L}}{{\partial a}} < 0$ for all $a \geqslant \mathop {\max }\limits_i \left| {{x_i}} \right|$. So, $\log L$ is a decreasing function on $[\mathop {\max }\limits_i \left| {{x_i}} \right|,\infty)$. Thus, $\log L$ attains maximum value on $[\mathop {\max }\limits_i \left| {{x_i}} \right|,\infty)$ at $a =\mathop {\max }\limits_i \left| {{x_i}} \right|$.

$\endgroup$
  • $\begingroup$ I think you should actually have $-n/a$ as the derivative of the log likelihood in the regime $a \geq \max |x_i|$. $\endgroup$ – Ian May 18 '16 at 13:33
  • $\begingroup$ @Ian: Yes. Thank you so much. I will edit $\endgroup$ – User3101 May 18 '16 at 13:34
0
$\begingroup$

An attempt at a full solution:

We have the likelihood function $$L(a \mid x_1, \dots, x_n) = \left(\dfrac{1}{2a}\right)^n\prod_{i=1}^{n}\mathbb{I}(x_i \in [-a, a])\text{.}$$ This is $0$ if at least one of the $x_1, \dots, x_n \notin [-a, a]$, which will obviously minimize $L$, rather than maximize $L$. Suppose, then, that $x_1, \dots, x_n \in [-a, a]$. Then the product of indicators is $1$, and we are left with $$L(a \mid x_1, \dots, x_n) = \left(\dfrac{1}{2a}\right)^n\text{.}$$ We need to actually use the data $x_1, \dots, x_n$ in calculating the MLE of $a$. Notice that $x_1, \dots, x_n \in [-a, a]$. We probably shouldn't choose $a$ to be negative - as this definitely wouldn't maximize $L$. So instead, notice that $|x_1|, \dots, |x_n| \in [0, a]$.

Obviously, $L$ is decreasing with respect to $a$. Using the intuition I developed before, one might (mistakenly) think to take $\hat{a} = \min_{i}|x_i|$. But the problem with this is that if $\hat{a} = \min_i|x_i|$, assuming we have at least two distinct $|x_i|$ values, they will lie outside of $[-\hat{a}, \hat{a}]$ and will thus give an indicator of $0$ (and therefore $L = 0$). The indicator is particularly important, as we had mentioned earlier that all of the $x_i$ values must be in $[-a, a]$. So, to avoid $L = 0$, we choose $\hat{a} = \max_i|x_i|$. Unfortunately, we can't choose $\hat{a}$ to be any smaller, or $L$ will be $0$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.