# Independent and identically distributed random variable

It is my first time learning probability theory, and if I understand correctly, the following is the meaning/motivation behind the definition of a random variable:

" A function's output is uniquely determined by its input. Random variable $$f$$ is a function defined on a sample space of a random phenomenon. Only after a realisation of the random phenomenon, we can know what is $$x$$ and hence $$f(x).$$ Additionaly, if the function is measurable, knowing the distribution of the underlying random phenomenon(on the sample space $$\Omega$$), one can understand the probability distribution of $$f(\Omega).$$ Thus, the so-called measurable functions are used to model this and are referred to as random variables.

Now let $$f$$ and $$g$$ two measurable functions defined on a measurable sample space $$(\Omega,\mu).$$ We say that $$f$$ and $$g$$ are independent random variables if the elements of sigma algebra generated by them are mutually independent.

What is the motivation behind this definition. ?

I understand that by independence of two events, we mean that occurrence of one does not change the chance of occurrence of others. Also, I am looking for some explicit real-world examples of a few functions that are all defined on the same sample space $$\Omega$$ to better understand the following:

1. Random variables are independent but not identically distributed.
2. Identically distributed but not independent.

Practical ones are appreciated. Thanks in advance.

• Two copies of the same variable will be identically distributed but not independent Commented Oct 1, 2022 at 12:05

## Motivation

Fix probability space $$(\Omega, \mathcal F, \mathbb P)$$.

Let's see how the definition of independence works in practice. Let $$f, g: \Omega \to \mathbb R$$ be measurable functions (random variables). Denote $$\mathcal A = \sigma(f) = \{ f^{-1}(E)| E\in \operatorname{Borel}(\mathbb R)\}$$ and $$\mathcal B = \sigma(g)$$. $$\sigma$$-algebras $$\mathcal A$$ and $$\mathcal B$$ are independent, if $$\forall A \in \mathcal A, B \in \mathcal B \quad \mathbb P (A\cap B) = \mathbb P(A)\mathbb P(B).$$ But elements of $$\mathcal A$$ are of the form $$f^{-1}(E)$$, for some Borel sets $$E$$. By convention, we write them $$\{f \in E \}$$. So our definition of independent variables is $$\forall E_1, E_2 \in \operatorname{Borel}(\mathbb R) \quad \mathbb P (f\in E_1, g\in E_2) = \mathbb P(f\in E_1)\mathbb P(g\in E_2).$$ where again we use some notation conventions (ommiting $$\{$$, $$\}$$ and replacing $$\cap$$ with $$,$$).

This shows what the definition is saying: any event that we could come up with which regards $$f$$, is independent with any event regarding $$g$$. This is fairly strong condition, which comes in handy in many situations.

## Examples

1. Independent but not identically distributed.

Very silly example could be $$f(\omega) = 1$$, and $$g(\omega) = 0$$ for all $$\omega \in \Omega$$. Those functions are measurable in any probability space, and all constant variables are independent (because $$\sigma(f) = \sigma(g) = \{\Omega, \emptyset\}$$). They obviously have different distributions.

Something more useful could be for example $$\Omega = \{0, 1, 2, 3\}$$, with $$f(x) = \begin{cases} 0 : \quad \text{when } x \leq 1\\ 1 : \quad \text{when } x > 1 \end{cases}, \quad g(x) = (x\!\!\!\! \mod 2) + 7$$ where we deal with classical probability. Here again, distributions are quite similar (only shifted by $$7$$), but not the same.

Product space

In general, given 2 random variables defined on different probability spaces we can "produce probability space in which they are independent". Given $$(\Omega_1, \mathcal F_1, \mathbb P_1), (\Omega_2, \mathcal F_2, \mathbb P_2), f:\Omega_1 \to \mathbb R$$ and $$g:\Omega_2 \to \mathbb R$$ we define $$\tilde f, \tilde g$$ on product space $$\Omega_1 \times \Omega_2$$ as $$\tilde f(\omega_1, \omega_2) = f(\omega_1), \quad \tilde g(\omega_1, \omega_2) = g(\omega_2).$$ It is quite easy to show, that $$\tilde f$$ has the same distribution as $$f$$, similarly for $$g$$. On top of that $$\tilde f$$ and $$\tilde g$$ are independent. So you could take any 2 variables in some model (different distribution or not) and make model in which they are independent.

2. Identically distributed but not independent.

Take any symmetrical distributed variable $$f$$. Define $$g = -f$$. Then $$f$$ and $$g$$ are not independent, but since distribution of $$f$$ was symmetrical, we have $$g \overset D = f$$.

Down-to-earth version of it can be for example $$f(x) = 2x-1$$ on $$\Omega = [0, 1]$$ with Lebesgue measure as probability. $$f$$ has uniform distribution $$U([-1, 1])$$. Then $$g(x) = 1 - 2x$$ has the same uniform distribution. Knowing value of $$f$$ gives immediately value of $$g$$, so they are not independent (you could check that generated $$\sigma$$-algebras are the same, which is contradiction to $$\sigma(f) \cap \sigma(g) = \{\Omega, \emptyset\}$$ for any independent $$f, g$$).

• thanks for your answer.But I was looking for some real world examples. Commented Oct 1, 2022 at 17:21
• What is real world example, if not functions you come across from time to time? Did you mean pre-math-model descriptions of life? Commented Oct 1, 2022 at 20:04
• Well I was looking for some examples say number of heads obtained in tossing a coin n times or something like that...which kind of motivates as to why these terminologies were coined..I really appreciate your answer. But could you please add few practical examples which are really used in mathematical modelling. Thanks in advance Commented Oct 1, 2022 at 20:30
• You toss a fair coin 10 times. Number of heads and number of tails are identically distributed, but not independent. Roll dice. Number rolled is independent from number of heads, but have different distribution. Commented Oct 1, 2022 at 20:37

Let $$(\Omega, \mathcal{F},\mathbb{P})$$ be a probability space and $$E_1, E_2 \subset \mathcal{F}.$$ Consider the random variables $$f_1=\mathbb{1}_{E_1}$$ and $$f_2=\mathbb{1}_{E_2}.$$ Now the following holds.

If $$E_1$$ and $$E_2$$ are not independent sets but $$\mathbb{P}(E_1)=\mathbb{P}(E_2)$$ then $$f_1$$ and $$f_2$$ are identically distributed but not independent random varibles.

If $$E_1$$ and $$E_2$$ are independent sets and $$\mathbb{P}(E_1)=\mathbb{P}(E_2)$$ then $$f_1$$ and $$f_2$$ are independent and identically distributed random varibles.

If $$E_1$$ and $$E_2$$ are independent sets but $$\mathbb{P}(E_1) \neq \mathbb{P}(E_2)$$ then $$f_1$$ and $$f_2$$ are independent but not identically distributed random varibles.

If $$E_1$$ and $$E_2$$ are not independent sets and $$\mathbb{P}(E_1) \neq \mathbb{P}(E_2)$$ then $$f_1$$ and $$f_2$$ are neither identically distributed nor independent random varibles.