Let $X_i$ be iid random variables with common density $f$ and distribution $F$. Let $Y_k = X_{(k)}$ be the k-th order statistics (that is, $Y_1 = X_{(1)} = \min(X_i)$ etc.). Show that the joint density of the order statistics is given by $$f_{X_{(1)}, X_{(2)}, \dots, X_{(n)}}(y_1, \dots, y_n) = \begin{cases} n! \prod_{i=1}^n f(y_i) & \text{for $y_1 < y_2 < \dots < y_n$} \\ 0& \text{otherwise} \end{cases}$$ Then show that $$f_{(k)}(y) = k {n \choose k} f(y) (1-F(y))^{n-k}F(y)^{k-1}$$

I am having trouble with this exercise. I can't seem solve it.

Thoughts on the first part

Well I thought let's start with $P(Y_1 \le y_1, \dots, Y_n \le y_n)$. This can be rewritten as $P(\exists i: X_i \le y_1, \exists i, j: X_i \le y_2, X_j \le y_2, \dots, \forall i: X_i \le y_n )$. This is ugly and the events are not even independent. I thought about using $P(A_1 \cap A_2 \cap \dots \cap A_n) = P(A_1) P(A_2 \mid A_1) \cdots P(A_n \mid A_1 \cap A_2 \cap \dots \cap A_{n-1})$ but I can't seem to get nowhere.

Thought on the second part

Given the first result, I thought about finding the marginal density, but this seems to be more complicated than I thought. I mean, $$f_{(k)}(y) = \int_{-\infty}^y \int_{y_1}^y\int_{y_2}^y \int_{y_{k-1}}^y \int_y^\infty \dots \int_{y_{n-1}}^\infty n! \prod_{i=1}^n f(y_i) dy_n \ \dots \ dy_1$$

which I doubt is correct and anyhow I am not able to solve all those integrals.

Can somebody offer some help? I don't think this way of doing it is feasible.


The probability that the $k$th order statistic is less than $x$ is the probability of the union of mutually exclusive events where at least $j \geq k$ of the samples are less than $x$ but the remaining $n-j$ samples are greater than $x$ -- where there are ${n}\choose{j}$ different combinations satisfying this condition. The probability of this union is a sum of probabilities over $j=k, \ldots,n$.

Assuming an iid sample of size $n$, the distribution function for $X_{(k)}$ is

$$\mathbb{P} (X_{(k)}\leq x) = \sum_{j=k}^{n} {{n}\choose{j}}[F(x)]^j[1-F(x)]^{n-j}$$

To get the density function, take the derivative of the distribution function with respect to $x$

$$f_{(k)}(x)\\=\sum^n_{j=k} {n \choose j}\cdot \left(jF(x)^{j-1}(1-F(x))^{n-j}f(x)-F(x)^{j}(n-j)(1-F(x))^{n-j-1}f(x) \right)\\=f(x)\left[\sum^n_{j=k} j{n \choose j} F(x)^{j-1}(1-F(x))^{n-j}-\sum^{n-1}_{j=k} (n-j){n \choose j} F(x)^{j}(1-F(x))^{n-j-1}\right].$$

Note that

$$j{n \choose j}=n{n-1 \choose j-1},\\ (n-j){n \choose j}=n{n-1 \choose j}.$$


$$f_{(k)}(x)\\=nf(x)\left[\sum^n_{j=k} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}-\sum^{n-1}_{j=k} {n-1 \choose j} F(x)^{j}(1-F(x))^{(n-1)-j}\right].$$

The first sum is

$$\sum^n_{j=k} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)} \\={n-1 \choose k-1}F(x)^{k-1}(1-F(x))^{n-k} + \sum^n_{j=k+1} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}$$

The second sum is

$$\sum^{n-1}_{j=k} {n-1 \choose j} F(x)^{j}(1-F(x))^{(n-1)-j)} \\=\sum^n_{j=k+1} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}.$$

The remaining sums cancel leaving

$$f_{(k)}(x) = n{n-1 \choose k-1}F(x)^{k-1}(1-F(x))^{n-k}f(x)$$

  • $\begingroup$ Well first of all thank you very much! I'm going to read it soon and get back to you. (note though that there may be some kind of off-by-one error: should be $n \choose k$ at the end. Anyhow, I am also very interested in the first part; if you have any ideas they would be most appreciated! Thank you again ;-) $\endgroup$ – Ant Mar 7 '15 at 11:51
  • $\begingroup$ No error because $n{n-1 \choose k-1} = k{n \choose k}$ $\endgroup$ – RRL Mar 7 '15 at 11:58
  • $\begingroup$ The derivation of the density is straightforward but messy. Another way is to show the distribution is equivalent to a beta-function integral, but that is also hard. $\endgroup$ – RRL Mar 7 '15 at 11:59
  • $\begingroup$ Of course.. I didn't notice you have a $n$ instead of a $k$ in the last expression! Thanks. Would you provide a hint about the first? It may be straightforward but I'll appreciate a nudge in the right direction :) $\endgroup$ – Ant Mar 7 '15 at 12:00
  • $\begingroup$ The joint density is more difficult. So I thought I'd do the easier part first. Ill get back to you. Thanks. $\endgroup$ – RRL Mar 7 '15 at 12:02

My answer may be not rigorous but it's simple. I derive the joint density without cumulative function as follows:
Let $\tau_{j}$ denote a permutation of $1,\dots,n$, and each permutation do not equals to each other. Thus, we have

$$ \begin{align*} & f_{X_{(1)}, X_{(2)}, \dots, X_{(n)}}(y_1, \dots, y_n) \\ = & \Pr \{X_{\tau_1(1)} =y_1, \dots, X_{\tau_1(n)} =y_n \} \\ & + \cdots \\ & + \Pr \{X_{\tau_{n!}(1)} =y_1, \dots, X_{\tau_{n!}(n)} =y_n \} \end{align*} $$

Each summand of the above equation is $\prod_{i=1}^n f(y_i)$ and the events are mutually exclusive.
Hence, it follows that $$ f_{X_{(1)}, X_{(2)}, \dots, X_{(n)}}(y_1, \dots, y_n) = \begin{cases} n! \prod_{i=1}^n f(y_i) & \text{for $y_1 < y_2 < \dots < y_n$} \\ 0& \text{otherwise} \end{cases} $$

Then the second statement can be obtained as follows:
To get $f_{(k)}(y)$, we first determine the positions of the random variables smaller than $X_{(k)}$, which has $n \choose k-1$ options. Then we determine the position of $X_{(k)}$ from the remaining positions, which has $n-k+1$ options.
Let $\delta$ denote a $k-1$ permutation of $n$. We have $$ \Pr\{X_{\delta_1} \le y, X_{\delta_2} \le y, \dots, X_{\delta_{k-1}} \le y \}=F(y)^{k-1}. $$ For a particular $X_i$, $\Pr\{X_i = y\} = f(y)$. Let $\zeta$ denote a $n-k$ permutation of $n$. We have $$ \Pr\{X_{\zeta_1} \ge y, X_{\zeta_2} \ge y, \dots, X_{\zeta_{n-k}} \ge y \}=[1-F(y)]^{n-k}. $$ Recall that, $k {n \choose k}={n \choose k-1}(n-k+1)$.
Finally, multiplying the factors above gives the second statement $$ f_{(k)}(y) = k {n \choose k} F(y)^{k-1} f(y) [1-F(y)]^{n-k}. $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.