Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $\xi_n\in \Xi$ be a sequence of iid random variables with $n \in\mathbb N\cup\{0\}$, which we call a noise process. Construct a process $$ Z_{n+1} = f(Z_n,\xi_n)\quad(\star) $$ with $Z_0\in E$ and $f:E\times\Xi\to E$ is a measurable function of two variables. Clearly, $Z$ is a Markov process.

On the other hand, consider a Markov process $X$ on $E$ given by its transition kernel $P(x,A)$. Is it always possible to find a noise process (on some set $\Xi$) and a measurable function $f:E\times \Xi\to E$ such that given any initial condition (maybe random) $X_0\in E$ $$ \operatorname{Law}(X_1) = \operatorname{Law}(f(X_0,\xi_0)) $$

Briefly speaking, if any Markov process can be presented in the form $(\star)$?

share|cite|improve this question
up vote 3 down vote accepted

Yes, this is the representation of a discrete time Markov process as a randomized dynamical system as found in Proposition 8.6 (page 145) from Foundations of Modern Probability (2nd edition) by Olav Kallenberg:

Let $X$ be a process on $\mathbb{Z}_+$ with values in a Borel space $S$. Then $X$ is Markov iff there exist some measurable functions $f_1,f_2,\dots:S\times[0,1]\to S$ and iid $U(0,1)$ random variables $\xi_n$ independent of $X_0$ such that $X_n=f_n(X_{n-1},\xi_n)$ almost surely for all $n\in\mathbb{N}$. Here we may choose $f_1=f_2=\cdots =f$ iff $X$ is time homogeneous.

share|cite|improve this answer
+1! Nice answer, Byron! Hope you are doing well! You might have noticed I have asked a related question… – Tim Nov 18 '12 at 18:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.