0
$\begingroup$

A gambler bets on coin flips. With each flip, he wins $1$ dollar with probability $p$, and loses $1$ dollar with probability $1-p$. He starts with $2$ dollar and stops when he reaches either $0$ or $5$ dollar.

  1. What is the probability that the gambler has $\$5$ in the end?

  2. What is the expected number of coin flips he bets on?

Ok, so basically my assumption towards these question is that:

  • Since $p$ and $1-p$ have been specified, its a biased coin.
  • The steps are $0$ to $5$, and it starts from step $2$.

Could anyone please confirm or maybe write down your answers if this is correct? or if I have left out any answers could any experts please help me with this question? Your help is much appreciated.

$\endgroup$

1 Answer 1

0
$\begingroup$

The transition matrix for your absorbing Markov chain is

$$P= \begin{bmatrix} 0 & p & 0 &0&0& 1-p \\ 1-p & 0 & p & 0 & 0&0 \\ 0 & 1-p & 0 & p & 0&0 \\ 0 & 0 & 1-p & 0 & p & 0\\ 0 &0 & 0 & 0 & 1 & 0\\ 0 &0 & 0 & 0 & 0 & 1\\ \end{bmatrix} \begin{matrix} \dots$1\\ \dots$2\\ \dots$3\\ \dots$4\\ \dots$5\\ \dots$0\\ \end{matrix}$$

So

$$Q= \begin{bmatrix} 0 & p & 0 &0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 0 & 1-p & 0 \\ \end{bmatrix} $$

$$R= \begin{bmatrix} 0 & 1-p \\ 0 & 0 \\ 0 & 0 \\ p & 0\\ \end{bmatrix} $$

and

$$ \begin{align} N&=(I-Q)^{-1}\\ &=\left( \begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&0&1\\ \end{bmatrix}- \begin{bmatrix} 0 & p & 0 &0 \\ 1-p & 0 & p & 0 \\ 0 & 1-p & 0 & p \\ 0 & 0 & 1-p & 0 \\ \end{bmatrix}\right)^{-1}\\ &=\left( \begin{bmatrix} 1 & -p & 0 &0 \\ p-1 & 1 & -p & 0 \\ 0 & p-1 & 1 & -p \\ 0 & 0 & p-1 & 1 \\ \end{bmatrix} \right)^{-1} \end{align} $$

You can invert this yourself because I'm too tired and lazy.

The point is that $B=NR$ will give you the answer to part 1

And $t=N\mathbf{1}$ will give you the expected number of steps for each starting position - you want the second row value.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .