Wikpedia defines a discrete-time Markov chain as a sequnce of random variables $X_1, X_2, X_3, \ldots$ with the property that $$P(X_{n+1}=x| X_1=x_1, X_2=x_2, \ldots, X_n=x_n) = P(X_{n+1}=x_{n+1}| X_n=x_n)$$

if both conditional probabilities are well-defined. Here all the random variables $X_i$ take values in a countable set $S$.

I do not understand what is the meaning of the above definition. A random variable has its domain as a probability space. The definition makes no mention of any probability space. I checked K. L. Chung's Elementary Probability Theory, and there also the definition is stated in the same manner, wihtout any reference to a probability space.

Can somebody please address this? Thanks.

  • $\begingroup$ The probability space is there, just not mentioned. What do you think $P(X_n = x_n)$ means, for example? $\endgroup$ – mathworker21 Feb 25 '18 at 6:50
  • $\begingroup$ @mathworker21 That is why I asked the question. Is it guaranteed that there is a probability space on which such random variables exist. spaceisdarkgreen has answered this. $\endgroup$ – caffeinemachine Feb 25 '18 at 11:36

In simple contexts like this it's rare to actually need to explicitly define or talk about the probability space. We just trust (hopefully someone's actually done the work!) that a space exists that can support the requisite random variables, and do everything from there in terms of the RVs.

For instance, how many problems have you done with a premise like "let $X$ and $Y$ be independent standard normal random variables"? There's a probability space lurking behind here, no doubt, but it would be a waste of time to define it. We know that it's possible to build probability spaces on which a pair of random normals can be defined (for instance the probability space defined by the joint law of the random variables works... but we're by no means committed to it if we want to later consider a third RV $Z$).

It's known (but not exactly trivial) that a probability space for an arbitrary Markov chain can be constructed, but understanding this technicality doesn't really help you understand Markov chains very much, so it's sensible to skip in most contexts. The general theorem for the existence of stochastic processes is called the Daniell-Kolmogorov theorem and is very technical (see here). This special case can be done with a few tricks and isn't much harder than the usual trick for constructing probability spaces for iid sequences.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.