Suppose I have the following equilibrium probability distribution: $\vec π = ({2\over5} , {1\over5} , {3\over20},{1\over4})$, corresponding to states 0,1,2,3, respectively.From my possible states of {0,1,2,3}, my initial state is randomly determined by the equilibrium probability distribution $\vec π$. I am after the probability of being in state 2 after n processes.

My understanding is that $\vec π$ is telling me once I get to the distribution of the desired state e.g. either one of $({2\over5} , {1\over5} , {3\over20},{1\over4})$ , it will stay in that distribution forever. So given that state 2 has been allocated ${3\over20}$ from the beginning it will stay there forever or it will be there after n processes. My final answer being ${3\over 20}$. $\\$

Might not be the easiest description to follow but a clarification would be great.


1 Answer 1


For a Markov Chain, a discrete-time stochastic process, the following holds: $$x^{(n+1)} = x^{(n)} P$$ where $x^{(n)}$ is the probability distribution at time $n$, $x^{(n+1)}$ the probability distribution at time $n+1$ and $P$ the transition matrix with elements $p_{ij}$ describing the probability of the next state being $j$ given the current state is $i$.

By definition, the stationary distribution $\pi$ is a vector such that $$ \pi = \pi P$$ so once in the stationary distribution, the chain remains in this distribution. Therefore if the chain is initialised into the stationary distribution, the probability of being in state 2 at a later time is indeed $\frac{3}{20}$.

See also this worked example (with numerical values) in the Wikipedia article on Markov chains to help.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .