# How to find distribution of order statistic

Let $X_i$ be iid random variables with common density $f$ and distribution $F$. Let $Y_k = X_{(k)}$ be the k-th order statistics (that is, $Y_1 = X_{(1)} = \min(X_i)$ etc.). Show that the joint density of the order statistics is given by $$f_{X_{(1)}, X_{(2)}, \dots, X_{(n)}}(y_1, \dots, y_n) = \begin{cases} n! \prod_{i=1}^n f(y_i) & \text{for y_1 < y_2 < \dots < y_n} \\ 0& \text{otherwise} \end{cases}$$ Then show that $$f_{(k)}(y) = k {n \choose k} f(y) (1-F(y))^{n-k}F(y)^{k-1}$$

I am having trouble with this exercise. I can't seem solve it.

Thoughts on the first part

Well I thought let's start with $P(Y_1 \le y_1, \dots, Y_n \le y_n)$. This can be rewritten as $P(\exists i: X_i \le y_1, \exists i, j: X_i \le y_2, X_j \le y_2, \dots, \forall i: X_i \le y_n )$. This is ugly and the events are not even independent. I thought about using $P(A_1 \cap A_2 \cap \dots \cap A_n) = P(A_1) P(A_2 \mid A_1) \cdots P(A_n \mid A_1 \cap A_2 \cap \dots \cap A_{n-1})$ but I can't seem to get nowhere.

Thought on the second part

Given the first result, I thought about finding the marginal density, but this seems to be more complicated than I thought. I mean, $$f_{(k)}(y) = \int_{-\infty}^y \int_{y_1}^y\int_{y_2}^y \int_{y_{k-1}}^y \int_y^\infty \dots \int_{y_{n-1}}^\infty n! \prod_{i=1}^n f(y_i) dy_n \ \dots \ dy_1$$

which I doubt is correct and anyhow I am not able to solve all those integrals.

Can somebody offer some help? I don't think this way of doing it is feasible.

## 2 Answers

The probability that the $k$th order statistic is less than $x$ is the probability of the union of mutually exclusive events where at least $j \geq k$ of the samples are less than $x$ but the remaining $n-j$ samples are greater than $x$ -- where there are ${n}\choose{j}$ different combinations satisfying this condition. The probability of this union is a sum of probabilities over $j=k, \ldots,n$.

Assuming an iid sample of size $n$, the distribution function for $X_{(k)}$ is

$$\mathbb{P} (X_{(k)}\leq x) = \sum_{j=k}^{n} {{n}\choose{j}}[F(x)]^j[1-F(x)]^{n-j}$$

To get the density function, take the derivative of the distribution function with respect to $x$

$$f_{(k)}(x)\\=\sum^n_{j=k} {n \choose j}\cdot \left(jF(x)^{j-1}(1-F(x))^{n-j}f(x)-F(x)^{j}(n-j)(1-F(x))^{n-j-1}f(x) \right)\\=f(x)\left[\sum^n_{j=k} j{n \choose j} F(x)^{j-1}(1-F(x))^{n-j}-\sum^{n-1}_{j=k} (n-j){n \choose j} F(x)^{j}(1-F(x))^{n-j-1}\right].$$

Note that

$$j{n \choose j}=n{n-1 \choose j-1},\\ (n-j){n \choose j}=n{n-1 \choose j}.$$

Then

$$f_{(k)}(x)\\=nf(x)\left[\sum^n_{j=k} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}-\sum^{n-1}_{j=k} {n-1 \choose j} F(x)^{j}(1-F(x))^{(n-1)-j}\right].$$

The first sum is

$$\sum^n_{j=k} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)} \\={n-1 \choose k-1}F(x)^{k-1}(1-F(x))^{n-k} + \sum^n_{j=k+1} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}$$

The second sum is

$$\sum^{n-1}_{j=k} {n-1 \choose j} F(x)^{j}(1-F(x))^{(n-1)-j)} \\=\sum^n_{j=k+1} {n-1 \choose j-1} F(x)^{j-1}(1-F(x))^{(n-1)-(j-1)}.$$

The remaining sums cancel leaving

$$f_{(k)}(x) = n{n-1 \choose k-1}F(x)^{k-1}(1-F(x))^{n-k}f(x)$$

• Well first of all thank you very much! I'm going to read it soon and get back to you. (note though that there may be some kind of off-by-one error: should be $n \choose k$ at the end. Anyhow, I am also very interested in the first part; if you have any ideas they would be most appreciated! Thank you again ;-) – Ant Mar 7 '15 at 11:51
• No error because $n{n-1 \choose k-1} = k{n \choose k}$ – RRL Mar 7 '15 at 11:58
• The derivation of the density is straightforward but messy. Another way is to show the distribution is equivalent to a beta-function integral, but that is also hard. – RRL Mar 7 '15 at 11:59
• Of course.. I didn't notice you have a $n$ instead of a $k$ in the last expression! Thanks. Would you provide a hint about the first? It may be straightforward but I'll appreciate a nudge in the right direction :) – Ant Mar 7 '15 at 12:00
• The joint density is more difficult. So I thought I'd do the easier part first. Ill get back to you. Thanks. – RRL Mar 7 '15 at 12:02

My answer may be not rigorous but it's simple. I derive the joint density without cumulative function as follows:
Let $\tau_{j}$ denote a permutation of $1,\dots,n$, and each permutation do not equals to each other. Thus, we have

\begin{align*} & f_{X_{(1)}, X_{(2)}, \dots, X_{(n)}}(y_1, \dots, y_n) \\ = & \Pr \{X_{\tau_1(1)} =y_1, \dots, X_{\tau_1(n)} =y_n \} \\ & + \cdots \\ & + \Pr \{X_{\tau_{n!}(1)} =y_1, \dots, X_{\tau_{n!}(n)} =y_n \} \end{align*}

Each summand of the above equation is $\prod_{i=1}^n f(y_i)$ and the events are mutually exclusive.
Hence, it follows that $$f_{X_{(1)}, X_{(2)}, \dots, X_{(n)}}(y_1, \dots, y_n) = \begin{cases} n! \prod_{i=1}^n f(y_i) & \text{for y_1 < y_2 < \dots < y_n} \\ 0& \text{otherwise} \end{cases}$$

Then the second statement can be obtained as follows:
To get $f_{(k)}(y)$, we first determine the positions of the random variables smaller than $X_{(k)}$, which has $n \choose k-1$ options. Then we determine the position of $X_{(k)}$ from the remaining positions, which has $n-k+1$ options.
Let $\delta$ denote a $k-1$ permutation of $n$. We have $$\Pr\{X_{\delta_1} \le y, X_{\delta_2} \le y, \dots, X_{\delta_{k-1}} \le y \}=F(y)^{k-1}.$$ For a particular $X_i$, $\Pr\{X_i = y\} = f(y)$. Let $\zeta$ denote a $n-k$ permutation of $n$. We have $$\Pr\{X_{\zeta_1} \ge y, X_{\zeta_2} \ge y, \dots, X_{\zeta_{n-k}} \ge y \}=[1-F(y)]^{n-k}.$$ Recall that, $k {n \choose k}={n \choose k-1}(n-k+1)$.
Finally, multiplying the factors above gives the second statement $$f_{(k)}(y) = k {n \choose k} F(y)^{k-1} f(y) [1-F(y)]^{n-k}.$$