Maximum Likelihood Estimation with Order Statistics' PDF

Question

Say I have $n$ samples, $x_1,x_2,...,x_n$, from a population with PDF $f(x)$ and unknown parameter $\theta$, the usual way of calculating the Likelihood of $\theta$ is

$L(\theta)=\displaystyle\prod_{i=1}^nf(x_i)$

But also, I can find the PDF of the order statistics of $f(x)$, say $f_{(1)}(x),f_{(2)}(x),...,f_{(n)}(x)$, and since I have the values of $x_1,x_2,...,x_n$, I can sort them into order statistics $x_{(1)},x_{(2)},...,x_{(n)}$.

Now I define a new function of $\theta$, say $M$ (I'm just taking the alphabet next to L)

$M(\theta)=\displaystyle\prod_{i=1}^nf_{(i)}(x_{(i)})$

Intuitively, you can maximize the value of $M(\theta)$ with respect to $\theta$ to obtain the estimation of $\theta$, say $\hat{\theta}$. Because that's what you do with right-censored data by replacing the PDF with Survival function, in this case I replaced the PDF with their respective order statistics' PDF.

Would $M$ result in the same function as $L$? If not, how would the two functions compare in term of estimating the parameter $\theta$?

Answer 1

Instead of $M(\theta),$ which is the product of marginal distributions of the order statistics, it is more logical to work with their joint likelihood, because unlike the observations themselves, the order statistics are not independent.

The joint distribution of all the order statistics can be shown to be $$f(x_{(1)},\dots,x_{(n)}|\theta)=n!f_{\theta}(x_{(1)})f_{\theta}(x_{(2)})\dots f_{\theta}(x_{(n)})\mathbb{I}({x_{(1)}\le x_{(2)}\le\dots\le x_{(n)}}).$$

Now note that $(x_{(1)},\dots,x_{(n)})$ is simply a permutation of $(x_1,\dots,x_n)$ and hence $\displaystyle\prod_{i=1}^nf_{\theta}(x_{(i)})=\prod_{i=1}^nf_{\theta}(x_i).$ This means that basing our inference on the joint distribution of all the order statistics and that on the original observations is equivalent. The situation of course changes if we do not have complete information in either of the cases, which can happen for right censoring.