MLE of uniform distribution

Question

Now before I begin, I know this question has been asked multiple times but all the answers but I had so many questions of my own that I figured I should make a new question as my thoughts are different than previous answers.

Now I will ask the question first then explain my thoughts and troubles :)

We have a uniform distribution that has the following PDF: $\frac{1}{b-a}$. So far so good. So if $n$ observations our Maximum Likelihood Function is: $\mathcal{L}(a,b)=\frac{1}{(b-a)^n}$ if each of these observations are independent and identically (i.i.d.) distributed. Now, after taking the log of the likelihood and taking the derivative once with respect to $b$ and once with respect to $a$ we have the following:

The derivative with respect to $a$ is: $$\frac{n}{b-a}$$ and the derivative with respect to $b$ is: $$\frac{-n}{b-a}$$

Now if we try to set either of these derivatives to zero and try to maximize the function, it will not yield anything useful. My problem arises here. Lets us just focus on maximizing $b$. I have read on this website as well as other places that to maximize $\frac{-n}{b-a}$ we have to take the maximum observation? How does that make sense, I mean should we not use the lowest observation to maximize this function? Because in this particular case, $n$ and $a$ are constants so we can easily just plug some numbers in and see that as $b$ gets bigger, the function get smaller, so why would we want the maximum observation? Help would be greatly appreciated and please I am not mathematically or statistically inclined so please be gentle! Thanks :)

Answer 1

Don't try to take derivatives. Note that the density of the uniform distribution is

$$ \frac{1}{b-a} I(a<X<b),$$

where $I$ is the indicator function. So your likelihood function should be something like

$$\frac{1}{(b-a)^n} \prod_{i=1}^n I(a<X_i<b).$$

Now eyeball that formula and see how it varies with $a,b$. First draw it for $a=0$ as a function of $b$, then the end result will become apparent.

Answer 2

Mathematically you can explain it as follows. You want to take $b$ as small as possible to maximize your derivative. Ok, but can you take $b$ smaller as the largest value you observed in the sample? No! That would be a contradiction to the fact that your sample comes from the (unkonwn) interval $[a,b]$ Could it be bigger? Yes, but you want it as small as possible.

So the maximization of the derivative with respect to $b$ is under the constrain that $b$ should be as small as possible but at least equal to the largest value of the sample. So, the lowest possible value for $b$ is the maximum of your sample and you have it.

Answer 3

Assume your observations are $X_1, X_2, ..., X_n$. Order them ascending so that you have your ordered sample $$X_{(1)},\ldots,X_{(n)}$$ that is $X_{(n)}=\max_{1\le n}\{X_i\}$. Now ask yourself: This $X_i$ are between $a,b$ but you do not know which numbers are $a,b$. Still $b$ is the highest limit. And you have found in your sample as highest value the value $X_{(n)}$. Leaving the math aside, wouldn't $X_{(n)}$ be your best estimate for the "possible" (i.e. unkown) upper limit $b$? Similarly with $X_{(1)}$ and $a$.

For an illustrative example: You know that the length (in cm) of a pencil is equally likely to be every number betwenn $[a,b]$ according to the manufacturer (i.e uniform distribution), but you do not know $[a,b]$ and you want to estimate them. You take a sample of $100$ pencils and you find the following values ordered ascending: $$10.2,10.2,10.2,10.3,\ldots10.8,10.9,10.9,10.9$$ So obviously $a$ is at most $10.2$ and $b$ is at least $10.9$ (no pencil's length can be less than a and no pencil's length can be greater than $b$). So what is your estimate for $a$ and what is your estimate for $b$? The mathematics in the MLE approach lead to the same result as the above intuition, i.e that the pencil's lenghts range within $[X_{(1)},X_{(100)}]=[10.2,10.9].$