We know that ginen a (discrete time) markov chain $\{X_n\}$, its transition matrix is stochastic. Is the inverse true? I.e. given a stochastic matrix $P$, is there a (discrete time) markov chain $\{X_n\}$ having $P$ as its transition matrix?

Attempt. I believe so. We are dealing with an existence theorem, and my idea has been the use of Kolmogorov's extension theorem, since it guarantees the existance of a desired markov chain under certain properties. Am I on the right path?

Thanks for the help!

  • 2
    $\begingroup$ What is wrong with just defining the process to make transitions based on the current state (independent of history) and using the probabilities specified by $P$? Equivalently, take a random walk on a graph. Kolmogorov extension theorem seems like overkill here. $\endgroup$ – Michael Mar 2 '17 at 1:14
  • $\begingroup$ Thank you! That seems reasonable! $\endgroup$ – Nikolaos Skout Mar 3 '17 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.