# modified stochastic process

Is there any study of stochastic processes where the probability matrix (for a finite state process) is time dependent?

For example, probability I go from school to home is higher at night as compared to in the morning where it is lower.

-

Yes. Time dependent (or time inhomogeneous) Markov chains share some common features with time homogeneous Markov chains, but their theory is not as nice, so most textbooks pass over them with just a comment.

One natural example of a time dependent chain is Polya's Urn Model. Start with one red and one blue ball in an urn. At each time, pull out a ball at random and return it along with another ball of the same color. If $X(n)$ is the number of red marbles in the urn, then
\begin{eqnarray*} \mathbb{P}(X(n+1)=k+1\ |\ X(n)=k)&=& {k\over n+2}\cr \mathbb{P}(X(n+1)=k\ |\ X(n)=k)&=& 1-{k\over n+2} \end{eqnarray*}

Then $X(n)$ has the Markov property, but the transition probabilities change over time.

-
How would one go about analyzing such a process? – picakhu Apr 8 '11 at 0:26
@picakhu It depends what you want to know. For instance, the $n$ step transition probabilities for an inhomogeneous chain are obtained using the product $P_0\ P_1\ \cdots \ P_{n-1}$ rather than the single matrix $P^n$ as in the homogeneous case. – Byron Schmuland Apr 8 '11 at 0:38
Does the theory stop there or are there other interesting results that can be dealt with here. – picakhu Apr 8 '11 at 2:09

If talk about Markov chains I believe that you can find literature on this topic due to this fact that the non-homogeneous Markov chain with a finite state space cannot be reduced to the homogeneous one with finite state space.

For the continuous state space non-homogeneous Markov processes $X(t)$ on can construct the homogeneous Markov process $Y(t) = (t,X(t))$ "without loosing" the structure of the state space. This statement is usually provided in books to justify why they consider only homogeneous processes in further chapters. In fact it's not true - even using this trick the process $Y(t)$ has quite difficult structure.

On the other hand, if you consider an example with you school attendance, it seems to be periodic (I hope) - and then for your Markov chain you don't need the infinite time horizon, so this non-homogeneous Markov chain can be reduced to the homogeneous one with a finite number of states.

Finally, as mr. Schmuland have mentioned, a lot of difficulties come from the fact that you have to use $$\prod_{i=k+1}^{n}P_i$$ rather than $P^{n-k}$.

-