# Non absorbing markov chain. Average state occurrence

If P is a $3\times 3$ transition matrix. Every state has a chance of going to every other state including itself. Therefore this is not an absorbing markov chain. What I want to be able to calculate is the probability of me being in a particular state. I know how to caluclate it after n steps, but surely this can keep changing for higher values of n?

Thanks

• It does keep changing, but there is the concept of stationary distribution, which represents the long-term "proportion" of time that your process spends in each state.
– user42268
Feb 29, 2016 at 20:01
• Thank you, but how can I calculate that from matrix P? Feb 29, 2016 at 20:29
• Are you sure that you want to calculate the probably at step $n$, rather than the asymptotic probability (the probability at step $\infty$) Mar 2, 2016 at 4:59

Def:

Let $\mathbf{P}$ be the transition probability matrix of a homogeneous Markov chain. If there exists a probability vector $\mathbf{\hat{p}}$ such that

$$\mathbf{\hat{p}}\mathbf{P} = \mathbf{\hat{p}}$$

then $\mathbf{\hat{p}}$ is called the stationary distribution for the Markov chain.

A stationary distribution $\mathbf{\hat{p}}$ is a (left) eigenvector of $\mathbf{P}$ with eigenvalue $1$. Note that any nonzero multiple of $\mathbf{\hat{p}}$ is also an eigenvector of $\mathbf{P}$ but the stationary distribution $\mathbf{\hat{p}}$ is fixed by being a probability vector; that is, its components sum to one.

When you want to find the stationary distribution $\mathbf{\hat{p}}$ of your Markov chain with the $3 \times 3$ probability matrix you can solve the system of equations

$$\mathbf{\hat{p}}\mathbf{P} = \mathbf{\hat{p}}$$

with $\mathbf{P}$ = $\begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \\ \end{bmatrix}$

you get

$ap_1 + dp_2 + gp_3 = p_1$

$bp_1 + ep_2 + hp_3 = p_2$

$cp_1 + fp_2 + ip_3 = p_3$

And as $\mathbf{\hat{p}}$ is a probability vector it must also fulfill $p_1 + p_2 +p_3 = 1$.

Solving this you obtain the stationary distribution:

$\mathbf{\hat{p}}= [p_1 \:\: p_2 \:\: p_3]$