1
$\begingroup$

Now before I begin, I know this question has been asked multiple times but all the answers but I had so many questions of my own that I figured I should make a new question as my thoughts are different than previous answers.

Now I will ask the question first then explain my thoughts and troubles :)

We have a uniform distribution that has the following PDF: $\frac{1}{b-a}$. So far so good. So if $n$ observations our Maximum Likelihood Function is: $\mathcal{L}(a,b)=\frac{1}{(b-a)^n}$ if each of these observations are independent and identically (i.i.d.) distributed. Now, after taking the log of the likelihood and taking the derivative once with respect to $b$ and once with respect to $a$ we have the following:

The derivative with respect to $a$ is: $$\frac{n}{b-a}$$ and the derivative with respect to $b$ is: $$\frac{-n}{b-a}$$

Now if we try to set either of these derivatives to zero and try to maximize the function, it will not yield anything useful. My problem arises here. Lets us just focus on maximizing $b$. I have read on this website as well as other places that to maximize $\frac{-n}{b-a}$ we have to take the maximum observation? How does that make sense, I mean should we not use the lowest observation to maximize this function? Because in this particular case, $n$ and $a$ are constants so we can easily just plug some numbers in and see that as $b$ gets bigger, the function get smaller, so why would we want the maximum observation? Help would be greatly appreciated and please I am not mathematically or statistically inclined so please be gentle! Thanks :)

$\endgroup$
1

3 Answers 3

3
$\begingroup$

Don't try to take derivatives. Note that the density of the uniform distribution is

$$ \frac{1}{b-a} I(a<X<b),$$

where $I$ is the indicator function. So your likelihood function should be something like

$$\frac{1}{(b-a)^n} \prod_{i=1}^n I(a<X_i<b).$$

Now eyeball that formula and see how it varies with $a,b$. First draw it for $a=0$ as a function of $b$, then the end result will become apparent.

$\endgroup$
2
  • $\begingroup$ Thanks but could you be more explicit about the nature of $I$, the indicator function? I've never seen the PDF of a uniform like that. The way I have it in my notes is simply $\frac{1}{b-a}$ where $a<X<b$ without the $I$ in the equation! Thanks! $\endgroup$
    – nicefella
    Commented Feb 27, 2014 at 0:27
  • $\begingroup$ Same thing. The indicator function takes the value one if it's argument is true and zero, otherwise. So the density equals zero outside of [a,b]. Your original formula didn't take that last bit into account. $\endgroup$
    – JPi
    Commented Feb 27, 2014 at 0:34
1
$\begingroup$

Mathematically you can explain it as follows. You want to take $b$ as small as possible to maximize your derivative. Ok, but can you take $b$ smaller as the largest value you observed in the sample? No! That would be a contradiction to the fact that your sample comes from the (unkonwn) interval $[a,b]$ Could it be bigger? Yes, but you want it as small as possible.

So the maximization of the derivative with respect to $b$ is under the constrain that $b$ should be as small as possible but at least equal to the largest value of the sample. So, the lowest possible value for $b$ is the maximum of your sample and you have it.

$\endgroup$
1
  • $\begingroup$ Quite an excellent answer! Thanks so much, it is all cleared up now! $\endgroup$
    – nicefella
    Commented Feb 27, 2014 at 1:42
1
$\begingroup$

Assume your observations are $X_1, X_2, ..., X_n$. Order them ascending so that you have your ordered sample $$X_{(1)},\ldots,X_{(n)}$$ that is $X_{(n)}=\max_{1\le n}\{X_i\}$. Now ask yourself: This $X_i$ are between $a,b$ but you do not know which numbers are $a,b$. Still $b$ is the highest limit. And you have found in your sample as highest value the value $X_{(n)}$. Leaving the math aside, wouldn't $X_{(n)}$ be your best estimate for the "possible" (i.e. unkown) upper limit $b$? Similarly with $X_{(1)}$ and $a$.

For an illustrative example: You know that the length (in cm) of a pencil is equally likely to be every number betwenn $[a,b]$ according to the manufacturer (i.e uniform distribution), but you do not know $[a,b]$ and you want to estimate them. You take a sample of $100$ pencils and you find the following values ordered ascending: $$10.2,10.2,10.2,10.3,\ldots10.8,10.9,10.9,10.9$$ So obviously $a$ is at most $10.2$ and $b$ is at least $10.9$ (no pencil's length can be less than a and no pencil's length can be greater than $b$). So what is your estimate for $a$ and what is your estimate for $b$? The mathematics in the MLE approach lead to the same result as the above intuition, i.e that the pencil's lenghts range within $[X_{(1)},X_{(100)}]=[10.2,10.9].$

$\endgroup$
2
  • $\begingroup$ Thanks so much! Intuitively it makes complete since but mathematically, I'm not sure I understand, but this really makes the picture for clear intuitively! Thanks! $\endgroup$
    – nicefella
    Commented Feb 27, 2014 at 0:24
  • 1
    $\begingroup$ Your welcome. Yes, I avoided maths because you said you do not like them... What you say about the smallest value of $b$ in your formula of the derivative above is correct! You want $b$ as small as possible! But, what is the smallest "admissible" value of $b$? Can $b$ be less than the largest value I observed? In the above example can I take $b$ less than $10.9$? Of course not. Bigger? Yes. But according to your formula, we want $b$ as small as possible. So take $b$ equal to $X_{(n)}$. So mathematica $\endgroup$
    – Jimmy R.
    Commented Feb 27, 2014 at 0:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .