Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is there any study of stochastic processes where the probability matrix (for a finite state process) is time dependent?

For example, probability I go from school to home is higher at night as compared to in the morning where it is lower.

share|cite|improve this question
up vote 2 down vote accepted

Yes. Time dependent (or time inhomogeneous) Markov chains share some common features with time homogeneous Markov chains, but their theory is not as nice, so most textbooks pass over them with just a comment.

One natural example of a time dependent chain is Polya's Urn Model. Start with one red and one blue ball in an urn. At each time, pull out a ball at random and return it along with another ball of the same color. If $X(n)$ is the number of red marbles in the urn, then
\begin{eqnarray*} \mathbb{P}(X(n+1)=k+1\ |\ X(n)=k)&=& {k\over n+2}\cr \mathbb{P}(X(n+1)=k\ |\ X(n)=k)&=& 1-{k\over n+2} \end{eqnarray*}

Then $X(n)$ has the Markov property, but the transition probabilities change over time.

share|cite|improve this answer
How would one go about analyzing such a process? – picakhu Apr 8 '11 at 0:26
@picakhu It depends what you want to know. For instance, the $n$ step transition probabilities for an inhomogeneous chain are obtained using the product $P_0\ P_1\ \cdots \ P_{n-1}$ rather than the single matrix $P^n$ as in the homogeneous case. – Byron Schmuland Apr 8 '11 at 0:38
Does the theory stop there or are there other interesting results that can be dealt with here. – picakhu Apr 8 '11 at 2:09

If talk about Markov chains I believe that you can find literature on this topic due to this fact that the non-homogeneous Markov chain with a finite state space cannot be reduced to the homogeneous one with finite state space.

For the continuous state space non-homogeneous Markov processes $X(t)$ on can construct the homogeneous Markov process $Y(t) = (t,X(t))$ "without loosing" the structure of the state space. This statement is usually provided in books to justify why they consider only homogeneous processes in further chapters. In fact it's not true - even using this trick the process $Y(t)$ has quite difficult structure.

On the other hand, if you consider an example with you school attendance, it seems to be periodic (I hope) - and then for your Markov chain you don't need the infinite time horizon, so this non-homogeneous Markov chain can be reduced to the homogeneous one with a finite number of states.

Finally, as mr. Schmuland have mentioned, a lot of difficulties come from the fact that you have to use $$ \prod_{i=k+1}^{n}P_i $$ rather than $P^{n-k}$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.