# Are sorted i.i.d still i.i.d?

I need help with that question.

We have i.i.d. Let us sort the values in ascending order.Is new series is i.i.d?

From my opinion the answers is yes because function y that sort the x is only mapping from x, but im not sure. Can anybody correct my understanding?

• It's not at all clear to me what it means to sort a sequence of random variables. Because given two RVs $X$ and $Y$, none of the three $X\le Y$, $X=Y$ nor $X\ge Y$ need hold. May 22, 2021 at 18:39
• Do not deface your question please. May 23, 2021 at 7:51

No, unless $$X$$ is a constant (non-random). First of all, the $$y_i$$ are clearly dependent: you already know that $$y_2\geq y_1$$. Furthermore, they are not identically distributed: the later values are more likely to be large.

I assume what you mean is something like this:

Let the $$n > 1$$ random variables $$X_1, X_2, \dots, X_n$$ over $$\mathbb R$$ be independent and identically distributed, and let $$Y_i$$ for $$1 ≤ i ≤ n$$ denote the $$i$$-th smallest value in $$(X_1, X_2, \dots, X_n)$$. That is to say, $$Y_1 = \min_{1 ≤ i ≤ n}(X_i)$$ and $$Y_n = \max_{1 ≤ i ≤ n}(X_i)$$, and in general $$i ≤ j \implies Y_i ≤ Y_j$$. Are the random variables $$Y_1, Y_2, \dots, Y_n$$ independent and identically distributed?

The answer is that the order statistics $$Y_1, Y_2, \dots, Y_n$$ are generally neither independent nor identically distributed.

• They are not independent because we know that e.g. $$Y_1$$ can never be larger than $$Y_2$$.

• They are not identically distributed because $$Y_1$$ is more likely to take smaller values than $$Y_2$$.

For example, let $$n = 2$$ and let us choose an arbitrary constant threshold $$c$$ such that $$p = P(X_i ≤ c)$$ is strictly between $$0$$ and $$1$$. Then we have four possible cases for the values of $$X_1$$ and $$X_2$$, with their corresponding probabilities as follows:

$$\begin{array}{r|c|c|} & X_1 ≤ c & X_1 > c \\ \hline X_2 ≤ c & p^2 & p(1-p) \\ \hline X_2 > c & p(1-p) & (1-p)^2 \\ \hline \end{array}$$

However, the corresponding table for $$Y_1$$ and $$Y_2$$ instead looks like this:

$$\begin{array}{r|c|c|} & Y_1 ≤ c & Y_1 > c \\ \hline Y_2 ≤ c & p^2 & 0 \\ \hline Y_2 > c & 2p(1-p) & (1-p)^2 \\ \hline \end{array}$$

This is because the four cases in the two tables are not all in one-to-one correspondence:

• $$Y_1 ≤ c \land Y_2 ≤ c \iff X_1 ≤ c \land X_2 ≤ c$$, and
• $$Y_1 > c \land Y_2 > c \iff X_1 > c \land X_2 > c$$, but
• $$Y_1 ≤ c \land Y_2 > c \iff (X_1 ≤ c \land X_2 > c) \lor (X_1 > c \land X_2 ≤ c)$$, and
• $$Y_1 > c \land Y_2 ≤ c$$ cannot happen.

From the table above we can also calculate the marginal probabilities $$P(Y_1 ≤ c) = p^2 + 2p(1-p) = 2p - p^2$$ and $$P(Y_2 ≤ c) = p^2$$. These can only be equal if $$p = 0$$ or $$p = 1$$, which was ruled out by the choice of $$c$$ above. Thus,

• $$Y_1$$ and $$Y_2$$ cannot be identically distributed, since $$P(Y_1 ≤ c) > P(Y_2 ≤ c)$$, and
• $$Y_1$$ and $$Y_2$$ also cannot be independent, since $$P(Y_1 ≤ c \mid Y_2 ≤ c) = 1 > P(Y_1 ≤ c)$$.

The only case where the argument above does not work is if we cannot choose a threshold $$c$$ satisfying the criterion that $$0 < P(X_i ≤ c) < 1$$, i.e. if the distribution of the $$X_i$$ variables is a degenerate distribution concentrated at a single point $$x$$. In that special case $$Y_1$$ and $$Y_2$$ are i.i.d., as $$Y_1 = Y_2 = x$$ almost surely.

The same argument can also be easily carried out, mutatis mutandis, for $$n > 2$$ and for any pair of indices $$1 ≤ i < j ≤ n$$; the only difference is that in that case the specific expressions in the second table become a bit more complicated.

Thus, for any $$n > 1$$, the only case where $$Y_1, Y_2, \dots, Y_n$$ are i.i.d. is if there is a constant $$x$$ such that $$X_i = x$$ almost surely.

• I upvoted this answer as I think it is the clearest one. I do not disagree with @Dasherman answer, simply I think it misses the whole point of the question: a misunderstanding about what random variables are, how you order them, and the impact on the distributions. Thanks Ilmari ! May 23, 2021 at 14:39

It depends whether you know the values of the $$\{y_i\}$$ or not. The knowledge of the values makes the random variables not IID anymore. If you are curious about the case when you do know the values at ordering, check out the "order statistic". Deriving the laws is a bit cumbersome in my opinion but completly doable by any students knowing probability.

https://en.wikipedia.org/wiki/Order_statistic

However, if you do not know the values at ordering time, then yes you can rearrange the IID random variables as you wish yourself :)

• ??? Whether or not something is true cannot depend on whether someone knows something... if two rvs are not iid then they're not iid, regardless of what anyone knows. May 22, 2021 at 10:36
• @DavidC.Ullrich perhaps there is a language barrier ? Probability is only about knowledge and information. Two rvs are independent by definition here (x1 x2) but if you evaluate them and then order them, then no they are not independent! Because you know the realisation of x1 and x2 already. It is the same as Brownian motion and Brownian bridge. Increments of a Brownian motion are independent of the future, but when you fix something in the future, (Brownian bridge) then yes they are dependent. I am unsure why the down votes because the question is exactly this, "order statistics". May 22, 2021 at 10:56
• If the only thing you know about the values is that $i \le j \implies y_i \le y_j$ (e.g. you pick the values independently at random from the same distribution and then, without looking at them, give them to someone else to sort and hand back to you), that's already enough to make the variables not i.i.d. Your answer seems to be implying that it's not. May 22, 2021 at 18:25
• @IlmariKaronen I think we all agree. My matter was the exact situation David describes in his comment under the question. How do you order X,Y,Z as random variables ? you can't in most cases, because there is no logical ordering for random variables in general. In that sense, you can reorder random variables the way you want. However, if you do it after evaluation $(i \leq j \implies y_i \leq y_j$) you get the order statistics. May 22, 2021 at 20:06
• @MarineGalantin: There is no conceptual problem with defining e.g. $A$ to be the smallest of $X$, $Y$, and $Z$, $B$ to be the second smallest of $X$, $Y$ and $Z$, and so on. You don't have to know the values of $X$, $Y$ and $Z$ to do that, although of course if you don't, you won't know for sure whether $A=X$ or $A=Y$ or $A=Z$ (although you'll know that at least one of those will hold by definition). But I agree that the disagreement here seems to be mainly one of terminology. I've added my own answer and tried to make it as explicit as possible in the hopes of clearing things up. May 23, 2021 at 12:34