# non-stationary Markov chain n-step

When I search for the long term behaviour of a stationary markov chain I just multiply the transition matrix with itself for the number of steps:

P(n) = P(0)^n.

But how do you go about doing it when the transition probabilites change with time (non-stationary)? Do I just multiply it as:

P(n)=P(0)P(1)P(2)..

where the the P:s change, or what do I do?

## 1 Answer

Yes, it's just a slightly more general version of the Chapman Kolmogorov equations; Let $P_{m,n}$ denote the matrix of probabilities of transitioning from a given state at time $m$ to another at time $n$. That is

$$[P_{m,n}]_{i,j}=P[X(n)=j|X(m)=i].$$

Using the law of alternatives

$$[P_{m,n}]_{i,j}=P[X(n)=j|X(m)=i] =$$

$$\sum_{k} P[X(n)=j|X(n-1)=k]P[X(n-1)=k|X(m)=i]=$$

$$[P_{n-1,n}P_{m,n-1}]_{i,j}.$$

Iterating the above we have that

$$[P_{m,n}]_{i,j}=[P_{n-1,n}P_{n-2,n-1}\dots P_{m,m+1}]_{ij}$$

In the "stationary" case you have that the one-step transition probabilities $P_{n,n+1}$ do not depend on when that step is taken, that is on $n$, that is $P_{n,n+1}=T$ for some $T$. So the above then reads

$$P_{m,n}=T^{n-m}.$$

Aside: People usually call this type of chain "non-homogenous" or "time-varying", "non-stationary" vs "stationary" usually means something different in this context. Namely, homogeneous means that the transition probabilities of the chain do not change in time. Stationary usually means that the law of $X_n$ does not change in time (with $n$). You can achieve the later for a homogenous chain by picking an appropriate initial law; can you guess what's got to be special about such initial laws (hint: the answer depends on $T$)?