Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

A gambler bets on coin flips. With each flip, he wins $1$ dollar with probability $p$, and loses $1$ dollar with probability $1-p$. He starts with $2$ dollar and stops when he reaches either $0$ or $5$ dollar.

  1. What is the probability that the gambler has $\$5$ in the end?

  2. What is the expected number of coin flips he bets on?

Ok, so basically my assumption towards these question is that:

  • Since $p$ and $1-p$ have been specified, its a biased coin.
  • The steps are $0$ to $5$, and it starts from step $2$.

Could anyone please confirm or maybe write down your answers if this is correct? or if I have left out any answers could any experts please help me with this question? Your help is much appreciated.

share|improve this question

1 Answer 1

up vote 0 down vote accepted

The transition matrix for your absorbing Markov chain is

$$P= \begin{bmatrix} 0 & p & 0 &0&0& 1-p \\ 1-p & 0 & p & 0 & 0&0 \\ 0 & 1-p & 0 & p & 0&0 \\ 0 & 0 & 1-p & 0 & p & 0\\ 0 &0 & 0 & 0 & 1 & 0\\ 0 &0 & 0 & 0 & 0 & 1\\ \end{bmatrix} \begin{matrix} \dots$1\\ \dots$2\\ \dots$3\\ \dots$4\\ \dots$5\\ \dots$0\\ \end{matrix}$$

So

$$Q= \begin{bmatrix} 0 & p & 0 &0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 0 & 1-p & 0 \\ \end{bmatrix} $$

$$R= \begin{bmatrix} 0 & 1-p \\ 0 & 0 \\ 0 & 0 \\ p & 0\\ \end{bmatrix} $$

and

$$ \begin{align} N&=(I-Q)^{-1}\\ &=\left( \begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&0&1\\ \end{bmatrix}- \begin{bmatrix} 0 & p & 0 &0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 0 & 1-p & 0 \\ \end{bmatrix}\right)^{-1}\\ &=\left( \begin{bmatrix} 1 & -p & 0 &0 \\ p-1 & 1 & -p & 0 \\ 0 & p-1 & 1 & -p \\ 0 & 0 & p-1 & 1 \\ \end{bmatrix} \right)^{-1} \end{align} $$

You can invert this yourself because I'm too tired and lazy.

The point is that $B=NR$ will give you the answer to part 1

And $t=N\mathbf{1}$ will give you the expected number of steps for each starting position - you want the second row value.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.