MLE of a uniform distribution

Question

I have a question about the MLE of the following distribution.

let $Y$ be a Uniform$(0,\theta)$ random variable, where $0<\theta<\infty$ and $\theta$ is to be estimated.

The first question asks me to write down the joing PDF of $Y_1...Y_n$ which I believe is the following: $$ \frac{1}{\theta^n}$$

The second question asks for the likelihood function which I think is:

$$Likelihood(y_1...y_n|\theta)= \frac{1}{\theta^n}$$

Now to find the ML Estimator of $\theta$, I would have to take the natural log and the derivative correct? After doing so I get the following:

$$-\frac{n}{\theta}$$ Unfortunately, setting this equation to $0$ does not yield anything valuable. How would I be able to find the ML Estimator and Estimate without using the derivative. The "answer" that I have in my notes says that I should "argue that making $\hat\theta$ equal to the largest observation maximizes the likelihood." Can someone please explain this argument in non-technical language?

Also, if we let $X$ denote the largest observation among $Y_1...Y_n$, how can we show that the PDF of $X$ is $$\frac{n}{\theta^n}x^{n-1}$$

How can one show that? Help would be greatly appreciated! Thanks so much!!

Answer 1

Since your question, "how would I find the PDF of $X$ if $X$ represents the largest observation" is a distinct question from finding the MLE of $\theta$, it warrants a separate answer.

For IID $X_1, X_2, \ldots, X_n \sim {\rm Uniform}(0,\theta)$, the last order statistic $$X_{(n)} = \max_i X_i$$ is the largest of the observed values in the sample. Consider its CDF: $$F_{X_{(n)}}(x) = \Pr[X_{(n)} \le x] = \Pr\left[ \bigcap_{i=1}^n X_i \le x \right].$$ This is because the largest of the observations is less than or equal to $x$ if and only if every observation is less than or equal to $x$. But since the observations are IID, it follows that $$F_{X_{(n)}}(x) = \prod_{i=1}^n \Pr[X_i \le x] = \begin{cases} 0 & x < 0 \\ (x/\theta)^n & 0 \le x \le \theta \\ 1 & x > \theta.\end{cases}$$ Consequently, the PDF of the last order statistic is $$f_{X_{(n)}}(x) = \frac{nx^{n-1}}{\theta^n}, \quad 0 \le x \le \theta.$$

As a further exercise, what is the PDF of the first order statistic (i.e., the minimum of the sample)?

Answer 2

Suppose I take a sample of size $n = 5$ observations of a ${\rm Uniform}(0,\theta)$ random variable, for which I know the value of the parameter $\theta$, but you do not. I tell you that the sample is $$X_1 = 3, X_2 = 5, X_3 = 1, X_4 = 5, X_5 = 2.$$ Your job is to now take that information and estimate $\theta$ in a way that maximizes $f(\boldsymbol X \mid \theta)$. Would you pick $\hat \theta = 4$? Of course not: the sample contains at least one observation of $5$, so it is impossible for the upper endpoint to be $4$.

Could you pick $\hat \theta = 1000$? Sure. But if the observations are uniform on $[0,1000]$, how likely is it to have observed $(3,5,1,5,2)$? Not very! So, your "most reasonable" estimate of $\theta$ should reflect this idea: the MLE statistic $\hat \theta$ is not necessarily correct (i.e., it is the true value of $\theta$), but it represents your best guess of the value of the parameter based on the information contained in the sample.

Now, what you have done by calculating the joint density and the likelihood function is show that if $\theta$ is greater than the largest of any of the observations (for otherwise, the likelihood is zero as we discussed earlier), then the likelihood is a monotonically decreasing function of $\theta$. In other words, choosing $\hat \theta = 6$ results in a bigger likelihood for our sample than $\hat \theta = 1000$. So, what is the smallest possible value you can choose? $\hat \theta = X_{(n)} = 5$.

Answer 3

When we define a function, we must specify the domain on which it is defined. When you picture a uniform distribution, the area under the curve must be 1. So we define the domain of the pdf so it satisfies this: $f(x) = 1/\theta$ for all $0 \leq x \leq \theta$. Suppose that $\theta$ is actually less than the largest observation, $Y_n$. Then $f(Y_n)$ should be $0$, but at the same time, since the outcome $Y_n$ was observed, it must also have positive probability. This is a contradiction!

So $\theta$ must be greater than the largest observation. But at the same time, we're trying to maximize our Likelihood function. If you graph your likelihood function, you'll realize that it is at its maximum at the least possible value for which $\theta$ is defined.