1
$\begingroup$

Based on a random sample (6.3, 1.8, 14.2, 7.6) use the method of maximum likelihood to estimate the maximum likelihoods for $\theta_1$ and $\theta_2$.

$$f_y(y;\theta_1, \theta_2) = \frac{1}{\theta_2- \theta_1} \;, \quad \theta_1 \le \theta_2$$

$$L(\theta_1, \theta_2) = \prod_\limits{i=1}^{n}\frac{1}{\theta_2-\theta_1} \\ = \frac{1}{(\theta_2- \theta_1)^n}\prod_\limits{i=1}^{n}1(\theta_1 \le y_i \le \theta_2) \\ = \frac{1}{(\theta_2- \theta_1)^n}\prod_\limits{i=1}^{n}1(\theta_1 \le y_i)1(y_i \le \theta_2) \\ \text{Let } T = \ln[L(\theta_1, \theta_2)] = -n \ln(\theta_2 - \theta_1) + \sum_\limits{i=1}^n\ln(1(\theta_1 \le \min_i(y_i))1(\max_i(y_i) \le \theta_2)) \\ \begin{cases} -\infty, & \text{if } \theta_1>\min_i(y_i) \text{or } \theta_2 < \max_i(y_i) \\ -n\ln(\theta_2 - \theta_1), & \text{otherwise} \end{cases} $$

>

now take the derivative with respect to one of them

$$\frac{\partial{T}}{\partial{\theta_2}} = \frac{-n}{\theta_2 - \theta_1} \\ = \frac{n}{\theta_1 - \theta_2}$$

To maximise this we want the numerator magnitude to be as small as possible, so we set $\theta_2 = \max_i(y_i)$

and for $\theta_1$

$$\frac{\partial{T}}{\partial{\theta_1}}=\frac{n}{\max_i(y_i) - \theta_1}$$

To maximise this, we want $\theta_1 = \min_i(y_i)$

This implies $\theta_1 = 1.8$ and $\theta_2 = 14.2$

If someone could check my correctness particularly around the indicator functions because I'm new to those and anything else you can see wrong in math or formatting.

Actually I think that stuff in yellow directly above is not right. I'm not equating the derivative to 0. I think I got the correct answer regardless. Probably more preferably is to look at the

$$-n\ln(\theta_2 - \theta_1)$$

And know to minimise the value in the brackets will maximise the $\ln[L(\theta_1, \theta_2)]$ function, and coming to the same result that I did illegitimately.

Implies that $\hat{\theta_1} = 1.8$ and $\hat{\theta_2}=14.2$

$\endgroup$
  • $\begingroup$ Again, differentiation is not valid here as in your previous questions on MLE with support depending on parameter (hence the indicator functions). $\endgroup$ – StubbornAtom Oct 5 '18 at 13:51
  • 1
    $\begingroup$ Given the sample $(y_1,y_2,\ldots,y_n)$, likelihood function of $\theta=(\theta_1,\theta_2)$ is \begin{align} L(\theta\mid y_1,y_2,\ldots,y_n)&=\frac{1}{(\theta_2-\theta_1)^n}\mathbf1_{\theta_1<y_1,\ldots,y_n<\theta_2} \\&=\frac{1}{(\theta_2-\theta_1)^n}\mathbf1_{\theta_1<\min y_i\,,\,\max y_i<\theta_2} \end{align} Clearly, the likelihood function is maximized when $\theta_2-\theta_1$ is minimized. Can you now justify why $L(\theta\mid y_1,\ldots,y_n)$ is maximized at $(\theta_1,\theta_2)=\left(\min\limits_{1\le i\le n} y_i,\max\limits_{1\le i\le n} y_i\right)$? $\endgroup$ – StubbornAtom Oct 5 '18 at 13:52
  • $\begingroup$ A more correct way of saying is that MLE of $(\theta_1,\theta_2)$ is $\left(\min\limits_{1\le i\le n} y_i,\max\limits_{1\le i\le n} y_i\right)$. Rather than MLE of $\theta_1$ is this and MLE of $\theta_2$ is that. It is a maximization problem in two variables $(\theta_1,\theta_2)$. $\endgroup$ – StubbornAtom Oct 5 '18 at 14:00
  • $\begingroup$ $\lim_{(\theta_2-\theta_1) \to 0} -n\ln(\theta_2 - \theta_1) = \infty$ @StubbornAtom Thankyou for your guidance. $\endgroup$ – Bucephalus Oct 5 '18 at 14:07
  • 2
    $\begingroup$ Possible duplicate of Stats - Likelihood function $\endgroup$ – StubbornAtom Oct 5 '18 at 18:16
0
$\begingroup$

You have $$ L(\theta_1,\theta_2) = \begin{cases} \dfrac 1 {(\theta_2-\theta_1)^n} & \text{if } \theta_1 \le \min\text{ and } \theta_2\ge\max, \\[6pt] \qquad 0 & \text{otherwise, i.e. if } \theta_1>\min \text{ or } \theta_2 < \max. \end{cases} $$ Since this function gets bigger as the two parameters get closer together, one maximizes it by putting them as close together as the constraints allow. I.e. $\widehat\theta_1= \min$ and $\widehat\theta_2=\max.$

$\endgroup$
  • 1
    $\begingroup$ That's very helpful @MichaelHardy thankyou. I have actually come here to make mistakes and try things that I think I understand, but I don't. I'm also relying on people like you to steer me in the right direction. I had this lecturer once, he was a professor of semi-conductors, etc. I remember him saying once, "Sometimes you think you understand something, and you go on operating with this assumption of understanding, and then ten years later the penny drops, and you go 'aaahhhh, that's how that works'". I remember you have been very helpful to me over the last year. I'm very grateful. Thanks $\endgroup$ – Bucephalus Oct 5 '18 at 18:05
  • $\begingroup$ That's good @MichaelHardy but the OP of that one doesn't make it easy to find with a heading like "Stats - Likelihood function" My question has a more specific header stating the operation, the distribution, and that it has two parameters. The alternative question doesn't identify the distribution either in the heading or in the question. I think mine would be easier to find for this problem. $\endgroup$ – Bucephalus Oct 5 '18 at 18:25
  • $\begingroup$ @Bucephalus I would have voted to close this as duplicate of the popular post here but the most voted answer is misleading. The post which I have voted for as the original, answers your question sufficiently. No need to fixate over this matter. $\endgroup$ – StubbornAtom Oct 5 '18 at 18:36
  • $\begingroup$ @Bucephalus : It said "two-parameter uniform distribution", but I would prefer to see it state explicitly that it means a uniform distribution on the interval $[\theta_1,\theta_2]. \qquad$ $\endgroup$ – Michael Hardy Oct 5 '18 at 18:38
  • $\begingroup$ @MichaelHardy changed as requested. $\endgroup$ – Bucephalus Oct 5 '18 at 18:47
3
$\begingroup$

This is more of an extended comment rather than an answer.

Sometimes a picture will tell you that the maximum likelihood estimates don't occur when derivatives are zero. Here is a plot of the likelihood surface for your data and model:

Likelihood surface

$\endgroup$
  • $\begingroup$ oh yeah, what program did you do that in @JimB Can you post the code? $\endgroup$ – Bucephalus Oct 5 '18 at 16:19
  • $\begingroup$ I read your profile, mathematica for sure. @JimB Is mathematica free? $\endgroup$ – Bucephalus Oct 5 '18 at 16:20
  • 1
    $\begingroup$ Mathematica is not free. You can google it to look for a price (as I suppose putting explicit advertisements here is frowned upon). I'll post the code later today when I have time. $\endgroup$ – JimB Oct 5 '18 at 16:32
  • $\begingroup$ It's ok, you don't have to. I do not think I would pursue it if it isn't free. Thanks @JimB $\endgroup$ – Bucephalus Oct 5 '18 at 16:33
  • $\begingroup$ Such a figure can also be constructed in R which is free (of course, "free" does not include the time it takes to learn it). (And many other "free" software titles will also create such figures.) $\endgroup$ – JimB Oct 5 '18 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.