0
$\begingroup$

I know this is probably very simple but I can't seem to solve this correctly.

X transitions to Y with a probability of 0.75, X transitions to X with a probability of 0.25 and X does not transition to Z.

Y transitions to X with a probability of 0.2, Y transitions to Y with a probability of 0.6, and Y transitions to Z with a probability of 0.2

Z transitions to Y with a probability of 0.2 and Z transitions to Z with a probability of 0.8

These probabilities render a stationary distribution with the following system of linear equations:

$$P_\infty(X) = \frac{1}{4}P_\infty(X) + \frac{1}{5}P_\infty(Y)$$ $$P_\infty(Z) = \frac{4}{5}P_\infty(Z) + \frac{1}{5}P_\infty(Y)$$ $$P_\infty(Y) = \frac{1}{5}P_\infty(Z) + \frac{3}{4}P_\infty(X) + \frac{3}{5}P_\infty(Y)$$

I can reduce the second equation to $\frac{1}{5}P_\infty(Z) = \frac{1}{5}P_\infty(Y) \rightarrow P_\infty(Z) = P_\infty(Y)$

I can reduce the first equation to $\frac{3}{4}P_\infty(X) = \frac{1}{5}P_\infty(Y) \rightarrow P_\infty(X) = \frac{4}{15}P_\infty(Y)$

I can rewrite the third equation as $\frac{1}{5}P_\infty(Y) + \frac{1}{5}P_\infty(Y) + \frac{3}{5}P_\infty(Y) = P_\infty(Y)$

However this leads me to $P_\infty(Y) = \frac{5}{5}P_\infty(Y)$, or $P_\infty(Y) = P_\infty(Y)$

Where did I go wrong?

$\endgroup$
2
$\begingroup$

There are no computational errors. The important observation is that writing those equations always results in a system that is rank-deficient. One equation is therefore replaced by the condition that the sum of all stationary probabilities should equal $1$. If you do that, you'll get $$ \left[\frac{2}{17}, \frac{15}{34}, \frac{15}{34} \right] $$ as solution, where indeed $P_\infty(Y) = P_\infty(Z)$.


Why are those equations always linearly dependent? If you write them in matrix form, you get $(Q-I)\,P_\infty = 0$, where $Q$ is the transition matrix of your Markov chain. Since $Q$ is a stochastic matrix, all its columns add to $1$. Subtracting the identity makes that sum $0$, which means that there exists a linear combination of the rows of $Q-I$ (namely, with unit coefficients) that is the $0$ vector. Hence $Q-I$ is not full rank.

Intuitively, the equations in $(Q-I)\,P_\infty = 0$ express flow conservation, and they say nothing about the amount of "flow." You need an additional equation to say that the "flow" is a probability distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.