# Two random variables from the same probability density function: how can they be different?

The definition of $X$ as a random variable according to Wiki is as follows:

$Let (\Omega, \mathcal{F}, P)$ be a probability space and $(E, > \mathcal{E})$ a measurable space. Then an $(E, \mathcal{E})$-valued random variable is a function $X\colon \Omega \to E$ which is $(\mathcal{F}, \mathcal{E})$-measurable. The latter means that, for every subset $B\in\mathcal{E}$, its preimage $X^{-1}(B)\in > \mathcal{F}$ where $X^{-1}(B) = \{\omega : X(\omega)\in B\}$. This definition enables us to measure any subset B in the target space by looking at its preimage, which by assumption is measurable.

And for real-valued random variables:

In this case the observation space is the real numbers. Recall, $(\Omega, \mathcal{F}, P)$ is the probability space. For real observation space, the function $X\colon \Omega \rightarrow > \mathbb{R}$ is a real-valued random variable if:

$\{ \omega : X(\omega) \le r \} \in \mathcal{F} \qquad \forall r \in > \mathbb{R}$.

Now in statistics and fields alike, they introduce random variables like $X \sim p(x)$ where $p(x)$ is a probability distribution. My question is if you say that $X\sim p(x)$ and $Y\sim p(x)$ how can these two represent two different random variables (like two different standard normal random variables) when they are sampled from the same $p(x)$, viz. how should you translate this to the formal measure theoretic definition that could differentiate between these two?

On the same $\Omega$, try $X$ uniform on $\{0,1\}$ and $Y=1-X$, then $\{X\ne Y\}=\Omega$.

Edit: Recall that in the probabilistic jargon, a random variable is just a measurable function, here $X:\Omega\to\{0,1\}$ and $Y:\Omega\to\{0,1\}$, that is, for every $\omega$ in $\Omega$, $X(\omega)=0$ or $X(\omega)=1$ and $Y(\omega)=0$ or $Y(\omega)=1$. A notation is that $\{X\ne Y\}=\{\omega\in\Omega\mid X(\omega)\ne Y(\omega)\}$. In the present case, $X(\omega)\ne Y(\omega)$ for every $\omega$ in $\Omega$ hence $\{X\ne Y\}=\Omega$.

Distributions, on the other hand, are probability measures on the target space $\{0,1\}$. Here the distribution $\mu$ of $X$ is uniform on $\{0,1\}$, that is, $\mu(\{0\})=\mu(\{1\})=\frac12$ since $P(X=0)=P(X=1)=\frac12$. Likewise, $P(Y=0)=P(Y=1)=\frac12$ hence $\mu$ is also the distribution of $Y$. Thus, $X$ and $Y$ can both be used to sample from $\mu$ although $X(\omega)=Y(\omega)$ happens for no $\omega$ in $\Omega$.

• What does the last expression mean? $\{X\neq Y\}=\Omega$?? I assume omega is a sample space. $X$ and $Y$'s defnition relies on the definition of $\Omega$, hence what you have written looks like a circular argument in terms of definitions. – Cupitor Mar 13 '14 at 20:55
• Ignoring the last part, by what you have written it seems that when we sample them from $p_x$ what we mean is we are sampling from the space of all $Z$'s where $Z:\Omega\to \{0,1\}$ where we keep the measure of the $\Omega$ the same. – Cupitor Mar 13 '14 at 21:41
• The Edit should answer these. – Did Mar 14 '14 at 5:31
• Thank you very much for additional explanation. Vote up. But I still didn't understand what I have asked in comment above after your edit and that is: So basically we can have two different random variables by changing the mapping that is applied from $\Omega$ to $\{0,1\}$ and unless they are the same (an also the measure on the sample and destination space is the same) the random variables are the same. Right? – Cupitor Mar 14 '14 at 12:25
• "...and unless the mappings are the same the random variables are NOT the same." Right. – Did Mar 14 '14 at 15:29

If you are simply told that $X$ and $Y$ share a probability distribution $p(x)$, you don't know that $X$ and $Y$ are different. Both are measurable functions from the sample space to the observation space with the given distribution, but otherwise they could be the same, independent, or different but correlated. What is often the case is that $X$ and $Y$ are independent. In that case, you can construct new variables with the same distribution as old ones by expanding the sample space. For instance, if $X:\Omega\rightarrow\mathbb{R}$ has a particular distribution, then $X_i:\Omega^N\rightarrow\mathbb{R}$, where $X_i(\omega_1,\omega_2,\ldots,\omega_N)\equiv X(\omega_i)$, are $N$ i.i.d. random variables with the same distribution as $X$ (if the sigma-algebra on the product space is appropriately constructed).

It may help to view this issue through the lens of push-forward measures.

Let $X$ and $Y$ be random variables on the probability space $(S,\mathcal F,\mathbb P)$ that take values in the measurable space $(S',\mathcal B)$.

Then the push-forward measure $\mu:= X_*\mathbb P$ is a measure on $S'$ that satisfies $(X_*\mathbb P)(B)=\mathbb P(X^{-1}(B))$ for every $B\in \mathcal B$. The push-forward measure $\nu:=Y_*\mathbb P$ is defined similarly.

Clearly the two push-forward measures are distinct when their construction is explicit, as above. But, but provided $\mu$ and $\nu$ assign the same probability to every set $B\in \mathcal B$, they are indistinguishable as measures on $(S',\mathcal B)$.

Here is a non-technical attempt. Random variable is a simply a numerical observation made on a result of a statistical experiment (a sample point). Your experiment could be simply finding the first moving car that you see in your street when you open your door in the morning. Car is a car, it is not a number. You can observe how much fuel it has in its tank, a numerical aspect of this sample point and so a random variable. Or observe how many passengers are there in the car, or how many miles it shows in the odometer. So, for a same statistical experiment there are many random variables are possible.Instead of the first car take the second car you notice and the fuel in its tank. This is a different random variable but it will have the same probability distribution as the r.v. of fuel of the first car you noticed.

TL;DR An analogy is that $x^2+5$ and $x^2+4$ have the same derivative $2x$ but are not the same function.

Elementary probability:

They don't teach this is in elementary probability, but random variables have an explicit representation known as the Skorokhod representation.

Basically, we never really know the formulas for a lot of the $X$'s. We know the $X$'s mainly from the $F_X(x)$'s. It's kinda like talking about $f(x)=x^2+c$'s through their common derivative $f'(x)=2x$: When is $f$ increasing? When $f' > 0$. We know that if $f$ is not unique given an $f'(x)$. We can do that through integration, or just construct an explicit example $f(x)=x^2+5$ and $f(x)=x^2+4$.

How we do similarly here in probability?

For example, consider $X \sim Be(p)$ where $P(X=0):=p$ and $P(X=1):=1-p$ (Usually, textbooks use $p$ for the $P(X=1)$).

If both of the following $X_i$'s satisfy $X \sim Be(p)$, then we've given explicit Bernoulli random variables that can never be the same, i.e. $X \sim Be(p)$ doesn't have a unique Skorokhod representation.

$$X_1(\omega) := 1_{(0,1-p)}(\omega) := 1_{A_1}(\omega)$$

$$X_2(\omega) := 1_{(p,1)}(\omega) := 1_{A_2}(\omega)$$

If $\omega=\frac{1-p}{2}$, then $X_1(\omega)=1$ while $X_2(\omega)=0$.

Let us try to compute the CDF of $X_i$:

$P(X_i(\omega) \le x)$ is 0 for $x<0$ and 1 for $x \ge 1$.

As for $0 \le x < 1$, define

$$P(X_i(\omega) \le x) = P(X_i(\omega) = 0) = P(1_{A_i}(\omega) = 0) = P(\omega \notin A_i) = 1 - P(\omega \in A_i)$$

We have our result if $P(\omega \in A_1) = P(\omega \in A_2) = 1-p$. Is it?

Okay so here, we need to need to make some kind of assumption to say that the interval $(p,1)$ is not only as probable as $(0,1-p)$ but also that probability of each interval is $1-p$. Clearly, the intervals have the same length, but does that mean they have the same probability? Furthermore, if they do, is it equal to That depends on how we define probabilities here. One such assumption is:

A uniformly distributed random variable $U$ on $(0,1)$ has Skorokhod representation $U(\omega) = \omega \sim Unif(0,1)$.

Hopefully this isn't circular, otherwise this half of the answer is nonsense.

Then $P(\omega \in A_i) = \frac{(1-p)-(0)}{1-0}$ or $= \frac{(1)-(p)}{1-0}$

$$P(\omega \in A_i) = \frac{1-p}{1-0} = 1-p$$

It can be shown that $$Y(\omega) = \omega \sim Unif(0,1)$$ for $\omega$ in $((0,1),\mathscr B(0,1),\mu)$ where $\mu$ is Lebesgue measure.
$$P(\omega \in A_i) = \mu(A_i) = l(A_i) = 1-p$$
where $l$ is length.