Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm trying to understand each of the above terms, and I'm having a lot of trouble deciphering the difference between them (note, my mathematics training isn't very strong - so please go easy on the jargon / expression with lots of colons and arrows between letters).

According to wikipeda:
A Markov Chain is a 'memoryless', 'random process'.
A Markov Process is a 'stochastic process', which exhibits the 'Markov property'.
The Markov property is the 'memorylessness' of a 'stochastic property'.
A Stochastic process is a 'random process', which is a collection of 'random variables'.
And finally, random variables are those determined by chance instead of other variables --- which seems to mean explicitly that they are memoryless.

Thus, it seems that stochastic process, random process, markov chain, and markov process are all the exact same thing... which is a collection of random variables, which are memory-less, which means they exhibit the Markov property.

Gah! Wikipedia mathematics is brutal...

share|improve this question
add comment

1 Answer

The difference between Markov chains and Markov processes is in the index set, chains have a discrete time, processes have (usually) continous.

Random variables are much like Guinea pigs, neither a pig, nor from Guinea. Random variables are functions (which are deterministic by definition). They are defined on probability space which most often denotes all possible outcomes of your experiment/model. In schools their value set is almost always a subset of $\mathbb{R}$.

Sequences of random variables don't need to be memoryless, e.g. sequences of random variables that denote some cummulative usually aren't memoryless. On the other hand, for example, sequences of independent identically distributed random variables do not depend on time at all, and so they have to be memoryless. Those two examples are something like extremes, where the next variable in the sequence depends on all of the previous (in the former example), or none of them (in the latter). The Markov property tells us, they may depend, but if they do, it is not much (e.g. in the case of discrete time, that is, Markov chains, it means that the next depends only on the current and nothing else).

I hope it explained something ;-)

share|improve this answer
    
Definitely, that is quite helpful! Thanks –  zhermes Dec 28 '12 at 0:49
    
Please modify sequences of independent random variables do not depend on time at all, which is misleading at the moment, and note that in the former example, it is not true in general that the next variable in the sequence depends on all of the previous. –  Did Jan 21 '13 at 11:34
    
@Did Fixed, thanks for pointing that out. –  dtldarek Jan 21 '13 at 11:46
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.