I was practicing some questions on transition probability matrices and I came up with this question.

You have 3 coins: A (Heads probability 0.2),B (Heads probability 0.4), C (Heads probability 0.6).Plan is to toss one of the 3 coins each minute. Start by tossing A. Subsequently if you toss Heads you coin A next minute. If you toss Tails, you chose at random with equal probability one of coins B or C for your next toss.

$X_n: $type of coin tossed on the nth minute.

Then the one steep transition probability matrix is: $$ \begin{matrix} 0.2 & 0.4 & 0.4 \\ 0.4 & 0.3 & 0.3 \\ 0.6 & 0.2 & 0.2 \\ \end{matrix} $$

a) Determine the asymptotic proportion of minutes you toss heads.
b) how could you compute the probability that you toss Heads on tosses 3 and 10.
My answer for a) is
$\pi_A *0.2 +\pi_B *0.4 +\pi_c* 0.6$ where $\pi_A,\pi_B,\pi_C$ are stationary distribution.
But the given answer is $\pi_A$.Can some one please tell me what is wrong with the way I did it.

For b) Is it P(tossing coin A on 4th toss and tossing coin on 11 th toss)=$P_{AA}^4*P_{AA} ^{11}$ ?

  • $\begingroup$ If the answer to a) is pi_A, this might indicate that a) asks for "the asymptotic proportion of minutes you toss" A, not "the asymptotic proportion of minutes you toss heads". $\endgroup$
    – Did
    May 2 '15 at 9:05
  • $\begingroup$ Both answers are OK. Accidentally $\pi_A *0.2 +\pi_B *0.4 +\pi_c* 0.6=\pi_A.$ See, my answer below. $\endgroup$
    – zoli
    May 2 '15 at 13:54
  • $\begingroup$ @Did, Surprisingly (not at all accidentally) the probability of heads equals the stationary probability of the "head" state however biased the coins are. $\endgroup$
    – zoli
    May 2 '15 at 15:39

Determining the stable (long term) state probabilities

The solutions of the following system of equations are the stationary state probabilities:

$$\begin{matrix}[\pi_A\ \pi_B \ \pi_C]\\ \\ \\ \end{matrix} \begin{bmatrix} 0.2 & 0.4 & 0.4 \\ 0.4 & 0.3 & 0.3 \\ 0.6 & 0.2 & 0.2 \\ \end{bmatrix} \begin{matrix}=[\pi_A\ \pi_B \ \pi_C]\\ \\ \\\end{matrix} $$ and $$ \pi_A+\pi_B+\pi_C=1 .$$


Here $\pi_A\not=1$. $\pi_A$ is a hypothetical quantity an intuitive definition of which (together with that of $\pi_B$,$\pi_C$) is as follows: "If we started the system choosing a state randomly with probabilities $\pi_A,\pi_B, \pi_C$ then the probability that the next state would be $A$, $B$, or $C$ would be $\pi_A,\ \pi_B, \pi_C$, respectively." Such stationary probabilities do not necessarily exists. You can read about the details here.


Having performed the vector-matrix multiplication we arrive at a redundant system of equations. After omitting the redundant equation and after some rearrangement we have

$$\begin{matrix} -0.8\pi_A & +& 0.4\pi_B & +& 0.6\pi_C &=0 \\ \ \ 0.4\pi_A & -& 0.7\pi_B& +& 0.2\pi_C&=0 \\ \pi_A&+&\pi_B&+& \pi_C&=1& \end{matrix}$$

The solution, with the help of Alpha, is $$\pi_A=\frac{5}{13}, \ \pi_B=\frac{4}{13},\ \pi_C=\frac{4}{13} .$$

Question a.

We can toss heads in each states, so $$P(\text{heads})=P(\text{heads}|\text{in state} A)\pi_A+P(\text{heads}|\text{in state}B)\pi_B+P(\text{heads}|\text{in state}C)\pi_C=\frac{2}{10}\frac{5}{13}+\frac{4}{10}\frac{4}{13}+\frac{6}{10}\frac{4}{13}=\frac{5}{13}=\pi_A.$$ That is, the result of the tossing is a head in $\frac{5}{13}100$% of the time on the long run.


It seemed to be accidental that


I wondered, if it was? So I took $p_A, p_B, \text{ and }p_C$ as the probabilities of the heads in the states $A, B, \text{ and } C$, respectively.

In order to get the stationary probabilities we have to solve the following system of equations:

$$\begin{matrix} p_A\pi_A & +& p_B\pi_B & +& p_C\pi_C &=\pi_A \\ \frac{1-p_A}{2}\pi_A & +& \frac{1-p_B}{2}\pi_B& +& \frac{1-p_C}{2}\pi_C&=\pi_B \\ \pi_A&+&\pi_B&+& \pi_C&=1& \end{matrix}.$$ The solutions are

$$ \begin{matrix} \pi_A=\frac{p_B+p_C}{-2p_A+p_B+p_C+2}\\ \pi_B=\frac{1-p_A}{-2p_A+p_B+p_C+2}\\ \pi_C=\frac{1-p_A}{-2p_A+p_B+p_C+2} \end{matrix}. $$ At this point, note that $p_A<1$. If $p_A$ was $1$ then the Markov chain would never get out of state $A$. Then, of course, $$1=P(\text{heads})=\pi_A.$$

If $p_A<1$ then the solutions above are valid and we may compute the probability of the heads in general. Watch this:

$$P(\text{heads})=\frac{(p_B+p_C)p_A+(1-p_A)p_B+(1-p_A)p_C}{-2p_A+p_B+p_C+2}=\pi_A \ !$$

Perhaps this is what the OOOOOP wanted to see.

Question b.

We have to go back to the initial state which is $A$. Now, the following vector-matrix product will give the probabilities that we end up in states $A$,$B$, or $C$ at the $\text{n}^{th}$ minute:

$$\begin{matrix}[1\ 0 \ 0]\\ \\ \\\end{matrix} \begin{bmatrix} 0.2 & 0.4 & 0.4 \\ 0.4 & 0.3 & 0.3 \\ 0.6 & 0.2 & 0.2 \\ \end{bmatrix}^n\begin{matrix}=[\pi_A^n\ \pi_B^n \ \pi_C^n]\\ \\ \\\end{matrix} $$

Finally, we have to repeat the steps performed during solution a. But this time we work with the results of the vector-matrix product above with $n=3$ and with $n=10$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.