Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I looked for a formal definition of Markov chain and was confused that all definitions I found restrict chain's state space to be countable. I don't understand purpose of such a restriction and I have feeling that it does not make any sense.

So my question is: can state space of a Markov chain be continuum? And if not then why?

Thanks in advance.

share|cite|improve this question
Since the state transitions themselves are countable, the state cannot enter more than a countable number of different states. So perhaps no generality is gained from having an uncountable state space. – MJD May 30 '12 at 14:16
@MarkDominus As I understand this, if I'm walking along the street I'm not allowed to model this in terms of Markov chain because world coordinates are continuous rather than discrete. – user32541 May 30 '12 at 14:24
I don't understand how you could model such a thing as a Markov process. What would the state transition be? You would need to have a continuous transition between states, and Markov processes are all about discrete transitions. – MJD May 30 '12 at 14:29
If I make a step and one and another and each step I'm in this or this point according to some probability distribution. Yep, I need continuous transition in this case, but it's much like Markov chain and I'm wondering whether it can be called Markov chain. – user32541 May 30 '12 at 14:41
"restrict chain's state space to be uncountable" -> sounds confusing. should "to be countable"? – leonbloy May 30 '12 at 14:49
up vote 4 down vote accepted

It seems you are thinking of a Markov process, rather than a Markov Chain. The Markov chain is usually defined as a Markov process that has a discrete (finite or countable) state space.

"Often, the term Markov chain is used to mean a Markov process which has a discrete (finite or countable) state-space" (ref)

A Markov process is (usually) a slightly more general thing, it's only required to exhibit the Markov property, which makes sense for uncountable states (and "time"), or more general supports. So, for example, the continuous random walk is normally considered a Markov process, but not a Markov Chain.

share|cite|improve this answer

Yes, it can. In some quarters the "chain" in Markov chain refers to the discreteness of the time parameter. (A notable exception is the work of K.L. Chung.) The evolution in time of a Markov chain $(X_0,X_1,X_2,\ldots)$ taking values in a measurable state space $(E, {\mathcal E})$ is governed by a one-step transition kernel $P(x,A)$, $x\in E$, $A\in{\mathcal E}$: $$ {\bf P}[ X_{n+1}\in A|X_0,X_1,\ldots,X_n] = P(X_n,A). $$ Two fine references for the subject are Markov Chains by D. Revuz and Markov Chains and Stochastic Stability by S. Meyn and R. Tweedie.

share|cite|improve this answer
Thanks for your reply, but it seems to be too complicated for me. I can't understand all that sigma-algebras =) Thanks for refs too. – user32541 May 30 '12 at 20:53

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.