# Given a stochastic matrix $P$, find a markov chain $\{X_n\}$ having $P$ as its transition matrix

We know that ginen a (discrete time) markov chain $\{X_n\}$, its transition matrix is stochastic. Is the inverse true? I.e. given a stochastic matrix $P$, is there a (discrete time) markov chain $\{X_n\}$ having $P$ as its transition matrix?

Attempt. I believe so. We are dealing with an existence theorem, and my idea has been the use of Kolmogorov's extension theorem, since it guarantees the existance of a desired markov chain under certain properties. Am I on the right path?

Thanks for the help!

• What is wrong with just defining the process to make transitions based on the current state (independent of history) and using the probabilities specified by $P$? Equivalently, take a random walk on a graph. Kolmogorov extension theorem seems like overkill here. – Michael Mar 2 '17 at 1:14
• Thank you! That seems reasonable! – Nikolaos Skout Mar 3 '17 at 15:43