Suppose we have an arbitrary probability space $(\Omega,\mathcal{F},\mathbb{P})$ and a sequence of random variables $X_n:\Omega\to\mathbb{R}$ such that the pushforward measures $(X_n)_*(\mathbb{P})$ converge weakly to a probability $\tilde{\mathbb{P}}$.
Can we always construct a random variable $X:\Omega\to\mathbb{R}$ such that $\tilde{\mathbb{P}}=X_*(\mathbb{P})$?

  • 4
    $\begingroup$ Certainly not. Take a countable $\Omega$ and approximate some continuous distribution. $\endgroup$ – fedja Oct 12 '13 at 1:46
  • $\begingroup$ Thank you, I thought of constructing such a counterexample, too. Could you exhibit an explicit counterexample? $\endgroup$ – Mizar Oct 12 '13 at 10:14
  • $\begingroup$ Well, I think I have just proved that you cannot proceed that way: if the $X_n$'s converge to a continuous distribution, the jumps in their distribution functions become (eventually) arbitrarily small, which contradicts the fact that $\Omega$ is countable.. $\endgroup$ – Mizar Oct 12 '13 at 11:04
  • $\begingroup$ Solved on MathOverflow, see here: mathoverflow.net/questions/145190/… $\endgroup$ – Mizar Oct 19 '13 at 22:38

If I didn't make a mistake, such a random variable must always exist.

Let $[-\infty,+\infty]$ be the set of real numbers along with the two infinities $\pm\infty$ in the order topology and with the Borel $\sigma$-algebra. Throughout this proof, random variables will be allowed to take values in $[\infty,+\infty]$.

Terminology: Suppose that for each $n\ge 1$, $\pi_n$ is a partition of some set fixed set $V$. We say that $\pi_n$ is an "increasingly fine" sequence iff $\pi_{n+1}$ is finer than $\pi_n$ for all $n$.

Recall the proposed proposition: Suppose $(\Omega,\Sigma,\mathbb{P})$ is a probability space, $X_n$ is a sequence of random variables, the cumulative distribution function of $X_n$ is $F_n$, and $F$ is some cumulative distribution function such that $F_n \to F$. (That is, at each point of continuity $x$ of $F$, $\lim_{n\to \infty} F_n(x) = F(x)$.)

Claim There is a random variable $X$ on the same probability space $(\Omega,\Sigma,\mathbb{P})$ with cumulative distribution function $F$.


For each integer $n\ge 1$, let $\mathcal{I}_n$ be the finite partition of $[-\infty,+\infty]$ consisting of the set $[-\infty,-n)$, the set $[n,+\infty]$, and the set $\left[i/2^n,(i+1)/2^n\right)$ for each integer $i\in [- 2^n n , 2^n n - 1]$. Loosely speaking, this is just a sequence of increasingly fine finite partitions that becomes arbitrarily fine at any fixed real number.

For each integer $n \ge 1$, let $\mathcal{P}_n$ be the finite partition of $\Omega$ consisting of sets of the form $X_1^{-1}(I_1)\cap \ldots \cap X_n^{-1}(I_n)$ where $I_1\in\mathcal{I}_1,\ldots,I_n\in\mathcal{I}_n$. Observe that $\mathcal{P}_n$ is a sequence of increasingly fine partitions of $\Omega$, each $\mathcal{P}_n$ is finite, and any $P\in\mathcal{P}_n$ is measurable.

We will consider the "limit" $\mathcal{P}$ of the sequence $\mathcal{P}_n$, which is the coarsest partition of $\Omega$ that is finer than any $\mathcal{P}_n$. The family $\mathcal{P}$ consists of all non-empty sets of the form $$ (1) \ \ \ \bigcap_{n\ge 1} P_n $$ for some sequence $P_n$ where $P_n\in\mathcal{P}_n$ for all $n$. That is, $$ \mathcal{P} = \left\{ \bigcap_{n\ge 1} P_n : P_n\in\mathcal{P}_n \mbox{ for all } n \right\}\setminus\{\emptyset\}. $$

Now let $\mathcal{A}$ consist of sets in $\mathcal{P}$ with positive measure: $$ \mathcal{A} = \{P\in\mathcal{P}: \mathbb{P}(P) > 0\}. $$ Loosely speaking, the sets in $\mathcal{A}$ are "atoms": nowhere in this proof will we encounter a nonempty proper subset of an atom. The number of atoms must be countable (otherwise, we'd have infinite measure), so we will write $$ \mathcal{A} = \{A_1, A_2,\ldots\}. $$

Next, define $A$ and $S$ as $$ A = \bigcup_{n\ge 1} A_n $$ and $$ S = \Omega \setminus A. $$ $A$ and $S$ are measurable. Basically, any random variable on $(\Omega,\Sigma,\mathbb{P})$ can be continuous on $S$ but $A$ will be responsible for jumps in the RV's cdf.

For each $n\ge 1$, construct the partition $\mathcal{S}_n$ of $S$ defined by $$ \mathcal{S}_n = \{ S \cap P : P\in \mathcal{P}_n \}. $$ Observe $\mathcal{S}_n$ is a sequence of increasingly fine finite partitions of $S$. Also, for any sequence $S_n$ with $S_n\in\mathcal{S}_n$, $$ (*) \ \ \ \ \mathbb{P}\left(\bigcap_{n\ge 1} S_n \right) = 0. $$ This is a key fact used below. To prove it, note $$ \mathbb{P}\left(\bigcap_{n\ge 1} S_n \right) = \mathbb{P}\left(S\cap \bigcap_{n\ge 1} P_n \right) $$ for some sequence $P_n\in\mathcal{n}$. If $\mathbb{P}\left(\bigcup_{n\ge 1} P_n\right) = 0$, then $(*)$ obviously holds. If $\bigcup_{n\ge 1} P_n$ has positive probability, then it must be $A_k$ for some $k$, by definition. But $S \cap A_k=\emptyset$, so $(*)$ again holds.

Now that some basic constructions and notation are in place, let's start proving things!

Lemma 1. For any $A_k$, there is an $\alpha_k\in [-\infty,+\infty]$ and a subsequence $X_{m_1},X_{m_2},\ldots$ of the sequence $X_1,X_2,\ldots$ such that $$ \lim_{n\to \infty} X_{m_n}(\omega) = \alpha_k \mbox{ for all } \omega\in A_k. $$

Proof. Fix $A_k$ and choose a single $x_n\in X_n(A_k)$ for each $n$. Since $[-\infty,+\infty]$ is compact, the sequence $x_1,x_2,\ldots$ must have a limit point $\alpha_k$ in $[-\infty,+\infty]$. Thus, there is a subsequence $x_{m_1}, x_{m_2},\ldots$ such that $$ (2)\ \ \lim_{i\to\infty} x_{m_i} = \alpha_k. $$ Recall from $(1)$ that there are sets $I_1\in\mathcal{I}_1,I_2\in\mathcal{I}_2,\ldots$ such that $$ A_k = \bigcap_{n\ge 1} X_n^{-1}(I_n), $$ so $x_n\in X_n(A_k)\subset I_n$ for all $n$. Suppose $\alpha_k$ is real and fix some $\omega\in A_k$. By $(2)$ and the construction of sets in $\mathcal{I}_n$, there has to be an $N$ such that $I_n$ is an interval of length $2^{-n}$ for all $n>N$. Thus, $$ |X_{m_n}(\omega) - x_{m_n}| \le 2^{-m_n} $$ for $n$ big enough. In light of this fact and (2), we have $X_{m_n}(\omega) \to \alpha_k$. A similar argument shows the same limit holds if $\alpha_k\in\{\pm\infty\}$.

Lemma 2. There are constants $\alpha_k\in[-\infty,+\infty]$ for $k\ge 1$ and a subsequence $X_{m_1},X_{m_2},\ldots$ of $X_1,X_2,\ldots$ such that for all $k$, $$ \lim_{n\to\infty} X_{m_n}(\omega) = \alpha_k \mbox{ for all } \omega\in A_k. $$

Proof Use the previous lemma and a diagonalization argument. First choose a subsequence $X_{11}, X_{12},\ldots$ of $X_1,X_2,\ldots$ such that $X_{1n}(\omega)\to\alpha_1$ for some $\alpha_1$ and all $\omega\in A_1$. Then choose a subsequence $X_{11}, X_{22}, X_{23}, \ldots$ of $X_{11},X_{12},\ldots$ such that $X_{2n}(\omega)\to\alpha_2$ for some $\alpha_2$ and all $\omega\in A_2$. At stage $k$, choose a subsequence $X_{11}, X_{22},\ldots, X_{kk},X_{k,k+1},\ldots$ such that $X_{kn}(\omega)\to \alpha_k$ for all $\omega\in A_k$. The subsequence $X_{11},X_{22},X_{33},\ldots$ is well-defined and has the desired property.

Important From now on, we will assume without loss of generality that for all $k$, $$ (3) \ \ \ \ \ \lim_{n\to\infty} X_n(\omega) = \alpha_k \mbox{ for all } \omega\in A_k. $$

We will now begin to construct a random variable $X$ with cumulative distribution function $F$. To do so, first define the following functions: \begin{equation} (4) \ \ \ F_A(t) = \sum_{n\ge 1,\ \ \alpha_n \le t} \alpha_n \\ (5) \ \ \ F_S(t) = F(t) - F_A(t) \end{equation} for all $t\in[-\infty,+\infty]$. We will construct random variables $X_A$ and $X_S$ such that $$ (6) \ \ \ \ \mathbb{P}(A \cap X_A \le t) = F_A(t) \\ (7) \ \ \ \ \mathbb{P}(S \cap X_S \le t) = F_S(t). $$ At that point, the proof will be complete, since we can let $X = 1_A X_A + 1_S X_S$. The cumulative distribution function of $X$ will then be $F_A + F_S = F$.

Let $X_A$ be a random variable with \begin{equation} X_A(\omega) = \alpha_k \mbox{ for all } k \mbox{ and for all } \omega\in A_k. \end{equation} The value of $X_A$ on $S$ is unimportant. One can set $X_A(S) = 0$, say, to ensure $X_A$ is measurable.

By construction of $X_A$, condition $(6)$ holds. Now now consider $X_S$.

Lemma 3 $F_S$ is increasing, right continuous, $F_S(-\infty) \ge 0$, and $F_S(\infty) = \mathbb{P}(S) $.

Proof By definition of $F_S$ and $F_A$, $F_S(-\infty) = 0$ and $F_S(+\infty) = \mathbb{P}(S)$. Consider $F(t)$. Since $X_n$ converges in distribution to the cumulative distribution function $F$, we can write $$ F(t) = \lim_{\epsilon\downarrow 0}\liminf_n \mathbb{P}(X_n \le t+\epsilon). $$ (Ask me why if you're not sure.) We can then write $$ F(t) = \lim_{\epsilon\downarrow 0}\liminf_n \mathbb{P}(A \cap X_n\le t+\epsilon) + \lim_{\epsilon\downarrow 0}\liminf_n \mathbb{P}(S \cap X_n\le t+\epsilon). $$ By $(3)$ and $(4)$, the first term is exactly $F_A(t)$. Thus, $$ F_S(t) = \lim_{\epsilon\downarrow 0}\liminf_n \mathbb{P}(S \cap X_n\le t+\epsilon). $$ This function is obviously increasing and right continuous.

I will use the following Lemma, which is really pretty easy to prove by direct construction. The proof is a little too tedious for me to include here. If you need more info, let me know.

Lemma 4 Suppose $U\subset\Omega$ is measurable and $\pi_1,\pi_2,\ldots$ is a sequence of increasingly fine finite partitions of $U$. Suppose for each $\epsilon>0$, there is an $n$ such that $\mathbb{P}(B)<\epsilon$ for all $B\in\pi_n$. Suppose $F:[-\infty,+\infty]\to[0,1]$ is increasing, right continuous, and $F(+\infty)=\mathbb{P}(U)$. Then there is a random variable $X$ on $(\Omega,\Sigma,\mathbb{P})$ such that $\mathbb{P}(U \cap X\le t) = F(t)$ for all $t\in[-\infty,+\infty]$.

Now apply Lemma 4 to construct $X_S$:

Lemma 5 There is a random variable $X_S$ such that $(7)$ holds, i.e., $\mathbb{P}(S \cap X_S \le t) = F_S(t)$.

Proof We will apply Lemma 4 with the set $S$, the sequence of partitions $\mathcal{S}_n$, and the function $F_S$. By Lemma 3, the conditions on $F_S$ required by Lemma 4 are met. We already observed $\mathcal{S}_n$ is a sequence of increasingly fine finite partitions. It only remains to prove that for any $\epsilon>0$, there is an $n$ such that $\mathbb{P}(B)<\epsilon$ for all $B\in\mathcal{S_n}$.

Fix $\epsilon>0$. For each $B\in\mathcal{S}_n$, let $c(B)$ represent the number of elements in the set $$ \{B'\in\mathcal{S}_{n+p} : B' \subset B,\ \ \mathbb{P}(B')>\epsilon, \ \ \ \ p\ge 0\}. $$ Informally, $c(B)$ is the number of elements "below" $B$ in the sequence of partitions $\mathcal{S}_n$ with probability greater than $\epsilon$. Observe $c(B)=0$ if and only if $\mathbb{P}(B)\le\epsilon$. Also, for any set $B\in\mathcal{S}_n$, if $\mathbb{P}(B) > \epsilon$, then $$ (8)\ \ \ c(B) = 1 + \sum_{B'\in\mathcal{S}_{n+1},\ \ B'\subset B} c(B'). $$

Fix some $B_0\in\mathcal{n}$ and suppose $c(B_0)=+\infty$. By $(8)$, there is a $B_1\in\mathcal{S}_{n+1}$ with $B_1\subset B_0$ and $c(B_1)=+\infty$. Continuing this process, we see there is a sequence $B_0 \supset B_1 \supset B_2 \supset\cdots$ such that $B_p\in\mathcal{S}_{n+p}$ and $\mathbb{P}(B_p) > \epsilon$ for all $p\ge 0$. But then $$ \mathbb{P}\left(\bigcap_{p\ge 0} B_{n+p}\right) \ge \epsilon > 0. $$ This contradicts $(*)$. Thus, $c(B)$ is finite for all $B\in \mathcal{S}_n$. By $(8)$, it is clear that there is an $n$ with $c(B)=0$ for all $B\in\mathcal{S}_n$. This proves the result.

  • $\begingroup$ Thank you, you followed the same way as on MathOverflow (see my last comment below the question), but you provided a thorough answer! Very clear and detailed, +1! $\endgroup$ – Mizar Oct 20 '13 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.