# Finding MLE given dependent observations from uniform distribution $U(0,\theta)$ [closed]

Suppose we are given random variables $$X_1,...,X_n$$ that are uniformly distributed on the interval $$[0,\theta]$$, with $$\theta >0$$ unknown. I know that if the $$X_1,...,X_n$$ are furthermore independent, we can conclude that the Maximum Likelihood Estimation is given as $$\hat{\theta} = \max\limits_i x_i$$.

I was wondering if the same conclusion also holds if the random variables $$X_1,...,X_n$$ are not independent. My main problem is that I am unable to determine the Likelihood function, which I do not know if it is possible if the $$X_i$$ are not independent.

I would appreciate any help.

• It’s the multivariate density Commented Jun 11 at 5:11
• Intuitively, it doesn't seem to me like the samples being dependent would change the MLE ? The maximizer of the likelihood should still be the max of the $x_i$ because anything less than the max of the $x_i$ is not possible. The likelihood only exists for $\hat\theta$ = the maximum of the $x_i$ or greater. Commented Jun 11 at 5:11
• @markleeds I understand that intuitively, we should obtain the same MLE, however, I was not able to prove it. I also understand that it should be zero is the $x_i$ are less than $0$ or greater then $\theta$. I was not able to show that the function is then decreasing for $\theta > \max\limits_i x_i$ Commented Jun 11 at 5:24
• There is no way to determine the likelihood function when you don't know the dependence structure of the $X_i$'s. Commented Jun 11 at 11:50
• @user007: Based on the answer's below, it looks my comment is totally wrong. My apologies. I would delete my comment but then your comment and stubbornatom's comment would not make sense. Commented Jun 11 at 12:51

The hypothesis that the sample maximum is the MLE, even if there are dependencies, is false. The MLE really depends on how the $$X_i$$ depend on each other. Consider this example:

$$X_1 \sim U(0, \theta)$$ $$X_n \sim U(0, \theta/2)\,\, \textit{if}\,\, x_1<\theta/2$$ $$X_n \sim U(\theta/2, \theta)\,\, \textit{if}\,\, x_1>\theta/2$$ for $$n>1$$. You can see that, marginally, each $$X_i$$ is $$U(0, \theta)$$, but they are not independent.

Now suppose that you observe $$x_1 = 0.1$$ and $$\max {x_i} = 0.9$$. The likelihood of $$\theta$$ being 0.9 is zero, since, being $$x_1<0.9/2$$, every subsequent $$x_i$$ should have lied in the $$[0, 0.45]$$ interval. The MLE here is 1.8, twice the maximum observed value.

• nicola: neat example. I didn't realize that dependence meant that the marginal distribution can be different for each $X_i$. Commented Jun 11 at 12:49
• Yes, I considered the marginally uniform a requirement for each $X_i$. The MLE depends on the joint probability function. Commented Jun 11 at 13:00
• @nicola Apologies, not sure what I was thinking. This works, +1 Commented Jun 11 at 14:25
• @nicola thank you for the simple and easy to understand answer Commented Jun 12 at 11:40

By definition, when $$X_1,\dots, X_n$$ are dependent, the MLE is $$X_{(n)}=\max\limits_i X_i$$ if

A: and $$X_1,\dots, X_n$$ have a joint pdf $$f$$

B: The parameter $$\theta$$ only affects the univariate marginal distributions of the joint distribution of $$(X_1,\dots, X_n)$$.

C: The joint pdf of $$\left ( \frac{X_1}{\theta},\dots, \frac{X_n}{\theta} \right )$$ is increasing in each of its arguments.

Condition A is considered so that the analysis of the likelihood function becomes easier in the following (the ML method can be used for a more general case where the density is defined as the Radon-Nikodym derivative of the probability distribution relative to a dominating measure other than Lebesgue measure). Condition B is used to make sure that the distribution of $$\left ( \frac{X_1}{\theta},\dots, \frac{X_n}{\theta} \right )$$ is independent of $$\theta$$.

Under these conditions, for $$0\le x_i \le \theta, i=1,\dots,n$$, we have

$$f \left ( x_1,\dots, x_n \right )=\left ( \frac{1}{\theta} \right )^n g \left ( \frac{x_1}{\theta},\dots, \frac{x_n}{\theta} \right )$$

where $$g: [0,1]^n\to \mathbb R_{\ge 0}$$ is the joint pdf of $$\left ( \frac{X_1}{\theta},\dots, \frac{X_n}{\theta} \right ).$$ Then, we have

$$l(\theta)=\left ( \frac{1}{\theta} \right )^n g \left ( \frac{x_1}{\theta},\dots, \frac{x_n}{\theta} \right ) I(\theta \ge x_{(n)})$$

where $$g$$ is independent of $$\theta$$ under condition B. Hence, under C, the MLE of $$\theta$$ is $$X_{(n)}$$.

Condition B is equivalent to that the copula $$C$$ associated with $$X_1,\dots, X_n$$ is independent of $$\theta$$, that is, in the following Sklar's representation:

$$F_{X_1,\dots, X_n} \left ( x_1,\dots, x_n \right )=C\left ( F_{X_1}(x_1),\dots, F_{X_n}(x_n) \right ),$$

the copula $$C$$ is independent of $$\theta$$. In this case, condition A is equivalent to that the copula $$C$$ has a joint pdf $$g$$, and condition C means that $$g$$ is increasing in each argument.

Note that in the independence case, for $$u\in [0,1]^n$$, we have $$C(u_1,\dots, u_n)=u_1×\cdots×u_n,$$ with $$g(u_1,\dots, u_n)=1,$$ and all the three conditions A, B, and C are satisfied.

Counterexample in lack of C:

For $$n=2$$, consider observations $$x_1=x_2=5$$ where the joint copula of $$\frac{X_1}{\theta}, \frac{X_2}{\theta}$$ is assumed to be the Frank copula [1] with parameter $$-2$$. Then, as you can see below from the plot of $$l(\theta)$$ [2], the maximum likelihood estimate of $$\theta$$ is a number greater than $$x_{(2)}=5$$.

• I am not sure A is required to use the maximum likelihood method: if $X_1 \sim U[0, \theta]$ and $X_i=X_1$ for all $i$ then there is still a likelihood $\big($proportional to $\frac1\theta\, \mathbf I_{[0\le X_1\le \theta]}\big)$, but no joint density. Commented Jun 11 at 17:07
• @Henry Thanks for your comment! To use ML method, we need the density of data, which can be a joint pdf (in A, I used this case), a joint pmf, or more generally the Radon-Nikodym derivative of the probability distribution relative to a dominating measure. For the example you gave, we may use the general case.
– Amir
Commented Jun 11 at 17:50
• @Amir thank you for the detailed answer. The reason I accepted nicola's answer is, that the counterexample is simpler Commented Jun 12 at 11:41
• @user007 You are welcome! I appreciate your kind consideration! I like Nicola's answer too. Actually, it could be guessed that the claim couldn't not hold generally, so I was not seeking to only present a counterexample. My answer provides sufficient conditions under which the claim holds. Hope weaker conditions will be presented.
– Amir
Commented Jun 12 at 12:10