# Coin Flipping Expectations

Thinking (as I often do to understand Probability) about coin flipping, I'm looking for someone to explain how - and I've tried to make this as arbitrary as possible - for a coin with probability $p$ of flipping a heads, we can investigate some of its probabilistic properties. We can restrict p to $0<p<1$ for convenience.

I've found the expected number of heads in $n$ flips to be $np$ and the variance for the number of heads to be $p(n-p)$ - if these are wrong, I'd appreciate some correction, though intuitively the former seems right at least.

Suppose then we have $Y$ heads in total. If we look at the flips individually, so say we define a function $X_i$, which takes value $1$ if the $i^\text{th}$ flip is heads, and $0$ if it's tails, how can we determine $\mathbb{E}[X_i|Y]$ (which I imagine we can re-write as $\mathbb{E}[X_1|Y]$) and how can we also determine$\mathbb{E}[Y|X_i]$?

Can we also find the expected numbers of flips before the first head?

I'm quite interested in seeing where these answers come from, so any help would be really useful. Thanks, MM.

EDIT 1

Variance for first case is $np(1-p)$ rather than $p(n-p)$.

-
The variance is $np(1-p)$ rather than $p(n-p)$ –  Henry Jan 27 '12 at 10:36
@Henry: You're right. Made a calculation error. Thanks. –  Mathmo Jan 27 '12 at 10:43

You're dealing with the binomial distribution. Wikipedia has means, variances and more for all the widely used distributions. Your mean is correct, but the variance is a bit off; it's $p(1-p)n$.

By linearity of expectation, the expected values of all the $X_i$ given $Y=y$ must add up to $y$, so $\mathbb E[X_i|Y=y]=y/n$.

The expected value of $Y$ given $X_i=x$ is just $x$ plus the expected value of the remaining $X_j$, which is $(n-1)p$, so $\mathbb E[Y|X_1=x]=x+(n-1)p$.

[Edit in response to the comment:]

The probability for the first heads to occur in the $k$-th flip is given by the geometric distribution $(1-p)^{k-1}p$. It has the finite mean $1/p$. However, there's nothing strange in general about a value being finite with probability $1$ yet having infinite expected value. This is the case for instance for $(1-p)^{-k}$ (where $k$ is again the number of flips until the first occurrence of heads).

-
My wording may be unclear, I meant that $\mathbb{E}[X_i|Y]=\mathbb{E}[X_1|Y]$. I was looking seperately for a way of dealing with $\mathbb{E}[Y|X_i]$. Thanks. –  Mathmo Jan 27 '12 at 10:45
@MiamiMaths: Sorry, no, the wording was clear enough; I misread it. –  joriki Jan 27 '12 at 10:47
Thanks for that, very useful. Can we find the expected number of flips until the first head? I've found an addendum in our notes that claims it is finite with probability $1$, but that its expected value is infinite. This seems strange. –  Mathmo Jan 27 '12 at 10:55
Thanks again, very thorough! –  Mathmo Jan 27 '12 at 11:37

Assuming the $X_i$ are independently and identically distributed, then indeed $\mathbb{E}[X_i|Y=y] = \mathbb{E}[X_1|Y=y]$, so if you know $n$ then $\mathbb{E}[X_i|Y=y] = \dfrac{y}{n}$.

Meanwhile $\mathbb E[Y|X_1=x]$ is not the same. If you know $n$ and $p$ then $$\mathbb E[Y|X_1=x] = x+(n-1)p.$$

Or you might have a prior distribution for $p$ and then be able to use Bayesian techniques. If the prior distribution is $\pi_0(p)$ then

$$E[Y|X_1=0] = 0+(n-1)\frac{\int_{p=0}^{1} p(1-p)\, \pi_0(p) \,dp} {\int_{p=0}^{1} (1-p)\, \pi_0(p) \,dp}$$

$$E[Y|X_1=1] = 1+(n-1)\frac{\int_{p=0}^{1} p^2\, \pi_0(p) \,dp} {\int_{p=0}^{1} p\, \pi_0(p) \,dp}$$

-