3
$\begingroup$

When I search for the long term behaviour of a stationary markov chain I just multiply the transition matrix with itself for the number of steps:

P(n) = P(0)^n.

But how do you go about doing it when the transition probabilites change with time (non-stationary)? Do I just multiply it as:

P(n)=P(0)P(1)P(2)..

where the the P:s change, or what do I do?

$\endgroup$
1
$\begingroup$

Yes, it's just a slightly more general version of the Chapman Kolmogorov equations; Let $P_{m,n}$ denote the matrix of probabilities of transitioning from a given state at time $m$ to another at time $n$. That is

$$[P_{m,n}]_{i,j}=P[X(n)=j|X(m)=i].$$

Using the law of alternatives

$$[P_{m,n}]_{i,j}=P[X(n)=j|X(m)=i] = $$

$$\sum_{k} P[X(n)=j|X(n-1)=k]P[X(n-1)=k|X(m)=i]=$$

$$[P_{n-1,n}P_{m,n-1}]_{i,j}.$$

Iterating the above we have that

$$[P_{m,n}]_{i,j}=[P_{n-1,n}P_{n-2,n-1}\dots P_{m,m+1}]_{ij}$$

In the "stationary" case you have that the one-step transition probabilities $P_{n,n+1}$ do not depend on when that step is taken, that is on $n$, that is $P_{n,n+1}=T$ for some $T$. So the above then reads

$$P_{m,n}=T^{n-m}.$$

Aside: People usually call this type of chain "non-homogenous" or "time-varying", "non-stationary" vs "stationary" usually means something different in this context. Namely, homogeneous means that the transition probabilities of the chain do not change in time. Stationary usually means that the law of $X_n$ does not change in time (with $n$). You can achieve the later for a homogenous chain by picking an appropriate initial law; can you guess what's got to be special about such initial laws (hint: the answer depends on $T$)?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.