Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Ok, I've already posted this a minute ago, but my text deleted itself while I was editing it :-( So next try:

Can you help me to understand the notation my professor uses to describe Markov processes? Let $(S,\Sigma)$ be a measurable space. We denote the Markov property as $$\mathbb{P}(X_{n+1}=x_{n+1}|X_0=x_0,\dots,X_n=x_n)=P(x_n,x_{n+1}),$$ so apparently $P$ denotes the transition matrix, here only dependent on $x_n$ and $x_{n+1}$. But then we spoke about the Chapman-Kolmogorov relation, which we denoted by $$P_{s+t}(x,B)=\int P_s (x,dy)P_t (y,B),$$ where $x\in S$ and $B\in \Sigma$. Have you seen this notation and can you explain to me what it means? (Unfortunately it was not explained in the lecture).

Thank you very much!

share|cite|improve this question
up vote 4 down vote accepted

This uses two conventions at once:

  • For every integer $t\geqslant0$, $P_t$ is the transition matrix of $t$ steps of the Markov chain with transition matrix $P$. Hence $P_0$ is the identity and $P_{t+1}=P_tP=PP_t$ for every $t\geqslant0$.
  • For every transition matrix $P$ and every $B$ in $\Sigma$, $P(x,B)=\sum\limits_{y\in B}P(x,y)$.

In the discrete setting which seems to be considered here, Chapman-Kolmogorov equation rather reads as $P_{s+t}(x,B)=\sum\limits_{y\in S}P_s(x,y)P_t(y,B)$.

The integral notation is often used to describe Markov processes on continuous state spaces (but the Markov property as you wrote it in your post definitely applies to the discrete setting).

Edit: When the state space is continuous, each $P_t(x,\cdot)$ is actually a probability measure on $(S,\Sigma)$, which is defined, for every $B$ in $\Sigma$, by $$ P_t(x,B)=\mathrm P(X_t\in B\mid X_0=x). $$ In other words, $P_t(x,\cdot)$ is the distribution of $X_t$ conditionally on the event $[X_0=x]$ hence, for every measurable bounded function $\varphi$ on $S$, $$ \mathrm E(\varphi(X_t)\mid X_0=x)=\int_S\varphi(y)P_t(x,\mathrm dy). $$ The Chapman-Kolmogorov notation you recall in your post means that $$ \mathrm P(X_{t+s}\in B\mid X_0=x)=\int_S\mathrm P(X_{t+s}\in B\mid X_t=y)\,P_t(x,\mathrm dy), $$ since, by stationarity, $$ \mathrm P(X_{t+s}\in B\mid X_t=y)=\mathrm P(X_s\in B\mid X_0=y)=P_s(y,B). $$

share|cite|improve this answer
Thank you for your answer! We are treating Markov process also in continuous time, only the introduction was in discrete time setting. But hence it is essential that I understand the integral notation. What does $P(x,dy)$ mean? – kelu Apr 28 '12 at 12:48
About your Edit: So the integral is just a Lebesgue integral with respect to the probability measure $P_t(x,\cdot)$? But why do we write $\int P_s(x,dy)P_t(y,B)$ and not $\int P_s(y,B)P_s(x,dy)$ as usually when writing a Lebesgue integral? – kelu Apr 28 '12 at 13:09
Or is that just in order to confuse me? :-) – kelu Apr 28 '12 at 13:15
About your first comment: the continuous time is not an issue for the notation $P(x,dy)$, rather the fact that the space is continuous. – Did Apr 28 '12 at 15:08
@rodie $$P(X_t\in A\mid X_s=x)$$ – Did Jan 12 '15 at 20:15

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.