# What makes random variables exchangeable and what is implied by exchangeability?

Consider a d-dimensional random vector$\ X=(X_j)$.$\ X$ is called exchangeable if$\ (X_1,...X_d)\mathop = \limits^d({X_{{j_1}}},...,{X_{{j_d}}})$ for any permutation$\ j_1,..., j_d$. If$\ X_j$ are iid,$\ X$ is exchangeable. The converse is false (correct me if I'm wrong). What can we say about just identically distributed $\ X_j$? What can we say about $\ X_j$ if we know that$\ X$ is exchangeable?

-

An easy way to build an exchangeable random vector $X=(X_k)_{1\leqslant k\leqslant n}$ is to start from an i.i.d. sequence $(Y_k)_{1\leqslant k\leqslant n}$ and to define, for every $k$, $X_k=\Phi(U,Y_k)$, where $U$ is a random variable independent on $(Y_k)_{1\leqslant k\leqslant n}$ and $\Phi$ a measurable function.

And, in some sense, this is the only way... as shown by de Finetti for infinite sequences and by Diaconis and Freedman for finite sequences.

-

Exchangeability implies being identically distributed, but not the reverse. An exchangeable sequence has extra symmetry.

For instance, below I've written the joint mass function of $(X,Y)$, where $X$ and $Y$ take values in $\{0,1,2\}$. Checking the marginals shows that they are identically distributed, but $$\mathbb{P}(X=0,Y=2)=1/7\neq \mathbb{P}(Y=0,X=2)=0/7.$$ That is, the random variables $X$ and $Y$ have the same distribution, but the random vectors $(X,Y)$ and $(Y,X)$ don't.

$$\begin{array}{c|ccc} x\backslash y&0&1&2\cr \hline 0&0/7&1/7&1/7\\ 1&2/7&0/7&1/7\\ 2&0/7&2/7&0/7\\ \end{array}$$

See here and here for further applications of exchangeability.

-

If the converse were true, then "i.i.d." and exchangeability would be the same thing, and instead of having a separate concept, "exchangeability", one would have a theorem saying there are these two equivalent characterizations of the concept.

Exchangeability is stronger than identical distribution. Note that $X_1\overset{d}{=}X_{j_1}$, so these two are identically distributed, and $j_1$ could be any index.

A couple of important examples of exchangeability:

• First choose $p\in(0,1)$ randomly and suppose it's uniformly distributed, then let $X_1,X_2,X_3,\ldots$ be conditionally i.i.d. given $p$, with $X_1 = \left.\begin{cases} 1 & \text{with probability }p \\ 0 & \text{with probability }1-p \end{cases} \right\}$. Then $X_1,X_2,X_3,\ldots$ are not independent. To see that, suppose you observe the first 20 of these are equal to $1$. That makes it quite probable that $p$ is near $1$, so that it's likely that $X_{21}$ is equal to $1$. But by symmetry in the way the distribution was described, they are exchangeable. De Finetti's theorem says every infinitely long exchangeable sequence is a mixture of i.i.d. sequences in this manner (but the distribution of $p$ need not be uniform).
• Suppose an urn contains 10 red and 15 green marbles, and $X_k$ the number of red marbles on the $k$th draw without replacement, for $k=1,\ldots,12$. If you see $10$ red marbles in the first $10$ draws, then you know the 11th one will be green, so they're not independent. But it's not hard to show they're exchangeable. Unlike the previous example, this one cannot be extended to an infinitely long sequence of exchangeable random variables.

You can't say anything about the distribution of $X_j$ if you know it's part of an exchangeable sequence, since any distribution at all can be in that role.

-