Consider the Markov Chain with state space $S=\{v,w,x,y,z\}$, transition matrix below:

$$\left[\begin{array}{cccccccccc} 0 & 0.4 & 0.6 & 0 & 0\\ 0 & 0.5 & 0.5 & 0 & 0\\ 0 & 0 & 0 & 0.1 & 0.9\\ 0 & 0 & 0& 0.2 & 0.8\\ 0.7 & 0 & 0.3 & 0 & 0 \end{array}\right]$$ (where the matrix rows/columns correspond to the states in alphabetical order), and $X_0=v$.

1) What is $P(X_1=x, X_2=z, X_3=v)$?

For this question, do I multiply all the states starting from state x -> state z -> state v?

which I get: $$0.9 \times 0.7 = 0.63$$ Am I on the right track?

2) Find the probability distribution of $X_4$.

Any hints for this question please?

3) Find $P(\{X_2=w\} \cup \{X_3=x\})$ and $P(X_2=x|X_4=z)$

Im completely getting lost. I'm assuming that the first one is finding the union of $X_2$ and $X_3$ and the second question is finding the probability of $X_2$ given $X_4$, I am not sure what these "X" values are. Anyone could you please help me or give me a hint on these questions?


First, let's agree on notation: by $X_n$ we mean the state at "time" $n$; and we are told that the initial state ($n=0$) is $X_0 = v$. The transition matrix, denotes $M_{i,j} = P(X_{n+1}=j | X_{n}=i)$ (the row corresponds to the 'before' state, the column to the 'after' state).

For the first question, you want to compute a particular transition path. But remember that you start from $X_0$ (it can help to draw a graph of the transitions), so you actually are computing:

$$P(X_1=x,X_2=z,X_3=v | X_0=v)=\\ =P(X_1=x | X_0=v) P(X_2=z|X_1=x) P(X_3=v|X_2=z) =\\ = 0.6 \times 0.9 \times 0.7$$

(The first equation is true because it's a Markov chain).

For the second, you need to compute the probabilities of arriving of each one of the five states at time $n=4$. You could do that by summing all the paths that start from $X_0=v$, but that would be painful. (In general, perhaps not so much in this case because there are few transitions with positive probability). A more elegant way is to recall that the 4-step transition probabilities is given by $M^4$. Once you compute that, you just take the first row.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.