From Wikipedia

In a Markov process, one assumes that $i_1 < \cdots < i_n$. Then, because of the Markov property, $$ p_{i_1,\ldots,i_n}(f_1,\ldots,f_n)=p_{i_1}(f_1)p_{i_2;i_1}(f_2\mid f_1)\cdots p_{i_n;i_{n-1}}(f_n\mid f_{n-1}), $$ where the conditional probability $p_{i;j}(f_i\mid f_j)$ is the transition probability between the times $i>j$. So, the Chapman–Kolmogorov equation takes the form $$ p_{i_3;i_1}(f_3\mid f_1)=\int_{-\infty}^\infty p_{i_3;i_2}(f_3\mid f_2)p_{i_2;i_1}(f_2\mid f_1) \, df_2. $$

I was wondering if the reverse is true. I.e., if a process satisfies the above form of the Chapman–Kolmogorov equation, will it be Markovian? Thanks and regards!

  • $\begingroup$ from the linked page: "In English, and informally," $\endgroup$ – Ilya Feb 14 '13 at 13:01

Yes, if the initial (unconditional) distribution holds for any subsequent propagation such that $$p_{i_2} (f_2) = \int_{-\infty}^\infty p_{i_2;i_1} (f_2|f_1) p_{i_1}(f_1) df_1 $$ then the Chapman-Kolmogorov uniquely defines a Markov process.

To see this, multiply the CK equation by the unconditional PDF of the conditioned variable, $p_{i_1} (f_1)$, to get the the joint density $$ p_{i_3,i_1} (f_1, f_3) = p_{i_1} (f_1) \cdot \int_{-\infty}^\infty p_{i_3;i_2} (f_3|f_2) \cdot p_{i_2;i_1}(f_2|f_1) \ df_2\\ $$ Differentiate by $f_2$. The lefthand side becomes a joint probability distribution of 3 variables (and the above joint probability distribution of 2 variables can now be seen as a marginal probability distribution), and the right-hand side yields the definition of a Markov process $$ p_{i_3,i_2, i_1} (f_1,f_2, f_3) = p_{i_3;i_2} (f_3|f_2) \cdot p_{i_2;i_1}(f_2|f_1) \cdot p_{i_1} (f_1) $$ This can be repeated to get the Markov definition to the $n^{th}$ order definition you gave.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.