A coin is tossed continuously until 2 heads or 2 tails appear respectively. Let the result of first toss is tail. The game is over when we get 2 heads respectively.

Determine the transition probability matrix.

Check my answer is correct or not.

Let $X_n$ denote the number of tail that appear. Let the state $S=\{0,1,2\}$.

So, the transition probability matrix is $$P= \begin{bmatrix} 1&0&0\\ 0&\dfrac{1}{2}&\dfrac{1}{2}\\ 0&0&1 \end{bmatrix}. $$


I think in this case you need to have 4 states: state 2 tails, state tail, state head and state 2 heads which I will call states 1,2,3 and 4 respectively.

$$P= \begin{bmatrix} 1&0&0&0\\ \dfrac{1}{2}&0&\dfrac{1}{2}&0\\ 0&\dfrac{1}{2}&0&\dfrac{1}{2}\\ 0&0&0&1 \end{bmatrix}. $$

If I understand you correctly you keep playing until you get either two heads or two tails. Then, states 1 and 4 are absorbing states.

Then when you are in state 2 (got a tail) then you can jump after the coin flip to either state 1 (two tail flips) or state 3 (one head).

If you start with a tail then your initial state probability vector $\pi$ would be

$$\pi= \begin{bmatrix} 0\\ 1\\ 0\\ 0 \end{bmatrix}. $$


Notice that $a_{i,j}$ element in matrix describes the probability of transition from state $i$ to $j$. Your matrix describes process in which '$0$' is absorbing state (which is obviously not true).

If the matrix shall describe number of consecutive tails (and the two tails is absorbing state) then the matrix is as follows: $$P= \begin{bmatrix} \dfrac{1}{2}&\dfrac{1}{2}&0\\ \dfrac{1}{2}&0&\dfrac{1}{2}\\ 0&0&1 \end{bmatrix}. $$

  • $\begingroup$ Why $P_{10}=\dfrac{1}{2}$? I think if we have first toss is tail then it is imposibble the number of tail next toss is $0$. $\endgroup$ Mar 14 '19 at 6:47
  • $\begingroup$ Rows/columns of your matrix are states? $\endgroup$
    – vermator
    Mar 14 '19 at 7:09
  • $\begingroup$ Yes. It is state. $\endgroup$ Mar 14 '19 at 7:10
  • $\begingroup$ Your matrix shall describe number of consecutive tails (which corresponds to your game), or number of tails in general? $\endgroup$
    – vermator
    Mar 14 '19 at 7:13
  • $\begingroup$ number of consecutive tails $\endgroup$ Mar 14 '19 at 7:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.