# Find the transition probability matrix. Check my answer.

A coin is tossed continuously until 2 heads or 2 tails appear respectively. Let the result of first toss is tail. The game is over when we get 2 heads respectively.

Determine the transition probability matrix.

Check my answer is correct or not.

Let $$X_n$$ denote the number of tail that appear. Let the state $$S=\{0,1,2\}$$.

So, the transition probability matrix is $$P= \begin{bmatrix} 1&0&0\\ 0&\dfrac{1}{2}&\dfrac{1}{2}\\ 0&0&1 \end{bmatrix}.$$

I think in this case you need to have 4 states: state 2 tails, state tail, state head and state 2 heads which I will call states 1,2,3 and 4 respectively.

$$P= \begin{bmatrix} 1&0&0&0\\ \dfrac{1}{2}&0&\dfrac{1}{2}&0\\ 0&\dfrac{1}{2}&0&\dfrac{1}{2}\\ 0&0&0&1 \end{bmatrix}.$$

If I understand you correctly you keep playing until you get either two heads or two tails. Then, states 1 and 4 are absorbing states.

Then when you are in state 2 (got a tail) then you can jump after the coin flip to either state 1 (two tail flips) or state 3 (one head).

If you start with a tail then your initial state probability vector $$\pi$$ would be

$$\pi= \begin{bmatrix} 0\\ 1\\ 0\\ 0 \end{bmatrix}.$$

Notice that $$a_{i,j}$$ element in matrix describes the probability of transition from state $$i$$ to $$j$$. Your matrix describes process in which '$$0$$' is absorbing state (which is obviously not true).

If the matrix shall describe number of consecutive tails (and the two tails is absorbing state) then the matrix is as follows: $$P= \begin{bmatrix} \dfrac{1}{2}&\dfrac{1}{2}&0\\ \dfrac{1}{2}&0&\dfrac{1}{2}\\ 0&0&1 \end{bmatrix}.$$

• Why $P_{10}=\dfrac{1}{2}$? I think if we have first toss is tail then it is imposibble the number of tail next toss is $0$. Mar 14 '19 at 6:47
• Rows/columns of your matrix are states? Mar 14 '19 at 7:09
• Yes. It is state. Mar 14 '19 at 7:10
• Your matrix shall describe number of consecutive tails (which corresponds to your game), or number of tails in general? Mar 14 '19 at 7:13
• number of consecutive tails Mar 14 '19 at 7:15