# Don't understand the *derivation* of geometrically distributed random variables

I don't understand the derivation of geometrically distributed random variables as done here (only the first $10$ lines - everything until exercise $2$ - are relevant for me).

Please bare with me, that I probably need a very formal and thorough explication, since I looked already at different website and books, which used explanations in a similar manner to the link I provided and I didn't understood any of them! (I have my "own" derivation, which makes sense to me, which can be found below)

In the course I took, they weren't rigorously defined, so I more or less tried to figure out what the professor did and it seems to me that what is written below is how we derived them. But the derivation in the link is different! And the central problem seems to be, that in the link there is a mathematical connection between the random variables $X_{k}:\left\{ s,f\right\} \rightarrow\mathbb{R},\ \omega\mapsto\omega$ (where our Bernoulli trail sample space is modeled by $\Omega:\left\{ s,f\right\}$, for " success'' and " failure'' and the whole experiment is $\Omega^{\mathbb{N}}$ and $k\in\mathbb{N}$), since there they somehow used that the $X_{k}$ are independent -- which doesn't make sense at all to me: If $s$ has probability $1>p>0$, then the sets $\left\{ X_{1}=s\right\}$ and $\left\{ X_{2}=s\right\}$ aren't independent, because $$P\left(\left\{ X_{1}=s\right\} \cap\left\{ X_{2}=s\right\} \right)=P\left\{ s\right\} =p\neq p^{2}=P\left(\left\{ X_{1}=s\right\} \right)P\left(\left\{ X_{2}=s\right\} \right).$$

Opposed to that, in my derivation there isn't a mathematical link between the random variable that has the geometric distribution and the $X_{k}$.

My derivation: Consider sequence of Bernoulli trials of finite length $n$ (where the space for each experiment is for example $\Omega:\left\{ s,f\right\}$). If $p$ is the probability to get success, $p(1-p)^{k-1}$ is the probability of having the first success in the $k$th trial, where $n\geqslant k\left(\geqslant0\right)$. Now we want to know what the probability is to have first success in the $k$th trial, where we don't have an upper bound $n$.

Since we don't have a way to define on the space of all infinite sequences, i.e. $\left(\omega_{1},\omega_{2},\ldots\right)$ mit $\omega_{i}\in\Omega$ a suitable probability distribution starting from our $\Omega$ (at least I don't know how to do it and using advanced measure theoretic machinery doesn't count, since I'm interested in hoe the geometric distribution is defined in " undergraduate'' setting). But we do know that $p(1-p)^{k-1}$ is independent of $n$, so we define a new $\hat{\Omega}:=\mathbb{N}$ und $\hat{p}\left(k\right):=p\left(1-p\right)^{k}$ we models the fact that we have first success in the $k$th trial. Then we can define $X:\hat{\Omega}\rightarrow\mathbb{R},\ x\mapsto x$ as the random variable whose distribution is which tells us, when we get the first succes - but mathematically there is no connection between this random variable, and the random variables $X_{k}:\left\{ s,f\right\} ^{n}\rightarrow\mathbb{R},\ \left(\omega_{1},\omega_{2},\ldots,\omega_{n}\right)\mapsto\omega_{k}$ or the random variables $X_{k}:\left\{ s,f\right\} \rightarrow\mathbb{R},\ \omega\mapsto\omega$.

-
Why do you think that $P\left(\left\{ X_{1}=s\right\} \cap\left\{ X_{2}=s\right\} \right)=P\left\{ s\right\} =p$? This is like saying that if you toss a fair coin twice, the probability of getting heads twice is $\frac12$, which is clearly false. –  Brian M. Scott Jan 1 '13 at 16:46
I'm working just formally here, and formally I have $$P\left(\left\{ X_{1}=s\right\} \cap\left\{ X_{2}=s\right\} \right)= P( \{ s\}\cap \{ s\}) =P\left\{ s\right\}=p.$$ At which "$=$" have I made the error ? –  user26698 Jan 1 '13 at 17:04
The first is wrong. In order to talk about $X_1$ and $X_2$, you must consider (at least) $\Omega^2$: you’re looking at $$P\left(\big(\{s\}\times\Omega\big)\cap\big(\Omega\times\{s\}\big)\right)\;.$$ –  Brian M. Scott Jan 1 '13 at 17:07
@BrianM.Scott Ah, I seem to get more to the heart of my problem: But why are then the $X_k$ defined as $X_k:\Omega \rightarrow \mathbb{R}$ and not $X_k:\Omega^n \rightarrow \mathbb{R}$ (by taking the projection and then applying the "previous" $X_k:\Omega \rightarrow \mathbb{R}$) ? Or is it a notation/convention to define $$P(\{X_1=s\}\cap \{X_2 =s\}):=P((\{s\} \times \Omega)\times (\Omega \times \{s\})) ?$$ Since I always understood generally $P(\{Y=s\) \cap \{Z =s\})$ to mean$P(Y^{-1}(s) \cap Z^{-1}(s) )$, such that we don't silently pass to a different sample space –  user26698 Jan 1 '13 at 17:34
Generally I find it weird that we first have $P:\mathcal{\Omega} \rightarrow[0,1]$ and then suddenly use $P$ to evaluate probabilities of a different sample space $\Omega^n$... –  user26698 Jan 1 '13 at 17:37

I’ve tried to summarize in this answer some of the discussion in the comments.

The root of your difficulty, I think, is confusion about the appropriate sample spaces. On the one hand you have the Bernoulli random variables $X_k$ for $k\in\Bbb Z^+$; each of them is a function on the sample space $\Omega$ taking values in $\{0,1\}$. We define $P(\{s\})=p$ and $P(\{f\})=1-p$, from which we have automatically that $$P_{X_k}(\{1\})=P(X_k=1)=P(\{s\})=p$$ and $$P_{X_k}(\{0\})=P(X_k=0)=P(\{f\})=1-p\;.$$

On the other hand you have the geometric random variable $N$, defined as the least $k\in\Bbb Z^+$ such that $X_k=1$. The experiment in this case is to perform an infinite sequence $\mathbf X=\langle X_1,X_2,X_3,\dots\rangle$ of independent Bernoulli trials, so the possible outcomes are the infinite sequences that are the points of the Cartesian product $\Omega^{\Bbb Z^+}$.

Note that independence of the Bernoulli trials is part of the definition of a geometric random variable. Thus, even without access to any measure theory we can naïvely argue that

\begin{align*} P_N(\{k\})&=P(N=k)\\ &=P(X_1=0~\&~X_2=0~\&~\ldots~\&~X_{k-1}=0~\&~X_k=1)\\ &\overset{*}=P(X_1=0)\cdot P(X_2=0)\cdot\ldots\cdot P(X_{k-1}=0)\cdot P(X_k=1)\\ &=(1-p)^{k-1}p\;, \end{align*}

using independence of the $X_i$ to justify the starred equality.

However, we can also look at the set of outcomes resulting in $N=k$: it’s

$$\left\{\omega\in\Omega^{\Bbb Z^+}:N(\omega)=k\right\}=\left\{\langle\omega_1,\omega_2,\omega_3,\dots\rangle\in\Omega^{\Bbb Z^+}:\omega_1=\ldots=\omega_{k-1}=f\text{ and }\omega_k=s\right\}\;.$$

If for each $n\in\Bbb Z^+$ we let $\Omega_n=\{s,f\}$, we can write this set as

$$\left\{\omega\in\Omega^{\Bbb Z^+}:N(\omega)=k\right\}=\{f\}^{k-1}\times\{s\}\times\prod_{n>k}\Omega_n\;,$$

a cylinder set in the product $\Omega^{\Bbb Z^+}=\prod_{n\ge 1}\Omega_n$.

Note that the probability that we naïvely assigned to this set actually is the product of the probabilities associated with the individual factors: all but finitely many of those are $1$, so in effect it’s a finite product. This assignment of probabilities to cylinder sets is in fact the starting point for constructing a probability measure on the product, and for this particular setting we need nothing more.

-
Thanks, that cleared up another chunk of unclarity. But I'm still confused (please notice that I'm only concerned with the formal details that don't seem to work together) about 1) $X_k=1$. I've seen this notation only as $P(X_k=1)$, where it means $P(\{\omega\in \Omega \mid X_k (\omega)=1\})$. So the least $k$ such that $\{\omega\in \Omega \mid X_k (\omega)=1\}$ doesn't make much sense. Shouldn't it have been $X_k(\{s\})=1$ ?2) How the sample space $\Omega^{\mathbb{Z}^+}$ (of X ? Is that a random variable ?) works together with the sample spaces of the $X_k$'s, (...) –  user26698 Jan 2 '13 at 15:45
@user26698: (1) It’s common to omit the argument of a random variable and just write (for instance) $X_k=1$; see remark (1) here. And no, it’s $$P\left(\left\{\omega\in\Omega^{\Bbb Z^+}:\omega_k=s\right\}\right)\;.$$ –  Brian M. Scott Jan 2 '13 at 16:22
@user26698: Individually each $X_k$ is a function on $\Omega$. In the context of the sequence of Bernoulli trials, however, what we’re actually looking at is not $X_k$ itself, but rather the composition $X_k\circ\pi_k$, where $\pi_k(\omega)=\omega_k$ is the canonical projection map from the product to the $k$-th factor. –  Brian M. Scott Jan 2 '13 at 16:40
@user26698: Yes. And that should help you make sense of that Independence between discrete random variables link: that example with the discrete random vector is the case $n=2$. –  Brian M. Scott Jan 2 '13 at 16:45
@user26698: No, it’s still an underlying assumption. I just meant to indicate that what is generally presented informally at the undergraduate level in terms of multiplying probabilities of independent events isn’t contradicted by what is done in a more sophisticated approach; the latter actually has pretty much the same starting point. And before I forget: you’re asking the right kinds of questions. –  Brian M. Scott Jan 2 '13 at 16:50