0
$\begingroup$

Suppose I have the following equilibrium probability distribution: $\vec π = ({2\over5} , {1\over5} , {3\over20},{1\over4})$, corresponding to states 0,1,2,3, respectively.From my possible states of {0,1,2,3}, my initial state is randomly determined by the equilibrium probability distribution $\vec π$. I am after the probability of being in state 2 after n processes.

My understanding is that $\vec π$ is telling me once I get to the distribution of the desired state e.g. either one of $({2\over5} , {1\over5} , {3\over20},{1\over4})$ , it will stay in that distribution forever. So given that state 2 has been allocated ${3\over20}$ from the beginning it will stay there forever or it will be there after n processes. My final answer being ${3\over 20}$. $\\$

Might not be the easiest description to follow but a clarification would be great.

$\endgroup$

1 Answer 1

2
$\begingroup$

For a Markov Chain, a discrete-time stochastic process, the following holds: $$x^{(n+1)} = x^{(n)} P$$ where $x^{(n)}$ is the probability distribution at time $n$, $x^{(n+1)}$ the probability distribution at time $n+1$ and $P$ the transition matrix with elements $p_{ij}$ describing the probability of the next state being $j$ given the current state is $i$.

By definition, the stationary distribution $\pi$ is a vector such that $$ \pi = \pi P$$ so once in the stationary distribution, the chain remains in this distribution. Therefore if the chain is initialised into the stationary distribution, the probability of being in state 2 at a later time is indeed $\frac{3}{20}$.

See also this worked example (with numerical values) in the Wikipedia article on Markov chains to help.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .