# modified stochastic process

Is there any study of stochastic processes where the probability matrix (for a finite state process) is time dependent?

For example, probability I go from school to home is higher at night as compared to in the morning where it is lower.

-

Yes. Time dependent (or time inhomogeneous) Markov chains share some common features with time homogeneous Markov chains, but their theory is not as nice, so most textbooks pass over them with just a comment.

One natural example of a time dependent chain is Polya's Urn Model. Start with one red and one blue ball in an urn. At each time, pull out a ball at random and return it along with another ball of the same color. If $X(n)$ is the number of red marbles in the urn, then
\begin{eqnarray*} \mathbb{P}(X(n+1)=k+1\ |\ X(n)=k)&=& {k\over n+2}\cr \mathbb{P}(X(n+1)=k\ |\ X(n)=k)&=& 1-{k\over n+2} \end{eqnarray*}

Then $X(n)$ has the Markov property, but the transition probabilities change over time.

-
How would one go about analyzing such a process? –  picakhu Apr 8 '11 at 0:26
@picakhu It depends what you want to know. For instance, the $n$ step transition probabilities for an inhomogeneous chain are obtained using the product $P_0\ P_1\ \cdots \ P_{n-1}$ rather than the single matrix $P^n$ as in the homogeneous case. –  Byron Schmuland Apr 8 '11 at 0:38
Does the theory stop there or are there other interesting results that can be dealt with here. –  picakhu Apr 8 '11 at 2:09
For the continuous state space non-homogeneous Markov processes $X(t)$ on can construct the homogeneous Markov process $Y(t) = (t,X(t))$ "without loosing" the structure of the state space. This statement is usually provided in books to justify why they consider only homogeneous processes in further chapters. In fact it's not true - even using this trick the process $Y(t)$ has quite difficult structure.
Finally, as mr. Schmuland have mentioned, a lot of difficulties come from the fact that you have to use $$\prod_{i=k+1}^{n}P_i$$ rather than $P^{n-k}$.