Suppose that:
Coin $1$ has probability of $0.7$ of coming up heads
Coin $2$ has probability of $0.6$ of coming up heads

If the coin flipped today comes up:
heads: then we select coin $1$ to flip tomorrow,
tails: then we select coin $2$ to flip tomorrow.

If the coin intially flipped is equally likely to be coin $1$ or coin $2$, then what is the probability that the coin flipped on the third day after the initial flip is coin $1$?
This is what they did:

Let $X_n$ denote the label of the coin that is flipped on the $n$th day after the initial flip. X_n is a Markov chain. The transition probability matrix is given by
$$P = \begin{pmatrix}0.7 & 0.3 \\ 0.6 & 0.4\end{pmatrix}$$
$$P^(2) = P^2 = \begin{pmatrix}0.67 & 0.33 \\ 0.66 & 0.34\end{pmatrix}\\P^(3) = P^3 = \begin{pmatrix} 0.667 & 0.333 \\ 0.666 & 0.334\end{pmatrix}.$$
Hence the required probability is
$$\mathbb{P}(X_3 = 1) \\= \mathbb{P}(X_3 = 1|X_0 = 1)\mathbb{P}(X_0 = 1) + \mathbb{P}(X_3 = 1|X_0 = 2)\mathbb{P}(X_0 = 2)\\ = \frac{1}{2}(P^{3}_{11} + P^{3}_{21}) = 0.6665.$$

I understand how they got the $P$, but I think I'm not interpreting it correctly. How did they go from the first "required probability" line to the first equality? Is this using $\mathbb{P}(X=x|Y=y) = \frac{\mathbb{P}(X=x \& Y=y)}{\mathbb{P}(Y=y)}$? I also don't see why they used those matrix components in the last line.


You're entirely correct about how they went from line 1 to line 2. Since the only possible values of $X_{0}$ are $1,2$, $$\mathbb{P}(X_{3}=1) = \mathbb{P}(X_{3}=1 \cap X_{0}=1) + \mathbb{P}(X_{3}=1 \cap X_{0}=2)$$ and from there, using $\mathbb{P}(A \cap B) \equiv \mathbb{P}(A|B)\mathbb{P}(B)$ the second line is obtained.

As for the last line, the matrix component $P_{ij}$ here represents the probability that we transition from state $i$ to state $j$ after one flip; similarly, $P^{3}_{ij}$ represents the probability from transitioning from state $i$ to state $j$ after 3 flips, which we can also write as $\mathbb{P}(X_{3}=j|X_{0}=i)$.

The whole point of the matrix formulation is so you can represent your probability state as a row vector $x$. In this case, initially $x = \left(\array{\frac{1}{2} &\frac{1}{2}}\right)$ since you are equally likely to start with either coin. By right-multiplying by $P^{n}$, the resulting row vector gives the probabilities of using each coin on the $n$th flip.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.