Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm learning stochastic process in college. How to understand Markov property?I'm curious about what is the power and validity of Markov property ?

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state, not on the sequence of events that preceded it.

The question is big and perhaps vague. To be specific and clear, I'd like to illustrate some aspects of the question.

Power:

  1. Why we need such property? Why is it essential and ubiquitous? What is the "philosophy" of such property?

Validity:

  1. Is this property an assumption when modelling? If it is, does it hold necessarily? If it does not hold, what's the influence?
share|improve this question

2 Answers 2

The underlying model is physical. In physics, everything is local (both in space and time). You wouldn't expect that something that happens on Mars right now would have an immediate influence on you. Similarly with something that happened million years ago. Moreover, it simplifies the modeling, since every vertex of the graph (if you work in the discrete setting) is connected only to few neighbors instead of everything else in the world.

Naturally, in particular empirical model it might turn out that having action at a distance simplifies things. Nevertheless, it is important to understand that any such model is a reduction of a local, physical, Markovian model.

share|improve this answer
    
Hmmm... something that happened million years ago might (and often does) have a pretty big immediate influence on me. –  Did Sep 7 '13 at 9:53
    
@Did only indirectly through a chain of events. The direct connection between any pair of those events only happens when they are infinitesimaly close together. –  Marek Sep 7 '13 at 9:56
    
Maybe so. Anyway your explanations do not seem to go to the heart of the matter (this is probably due, at least in part, to some rather imprecise analogies). –  Did Sep 7 '13 at 9:59
    
@Did I found you have several excellent answers related to Markov process which suggest you must be an expert in this field. I expect you can solve this problem when you're free:) –  John Hass Sep 7 '13 at 13:00
    
@Did: I'll be happy to listen to your explanation of the heart of the matter. The question was quite vague but still asked what sets Markov models apart from general models. I'm pretty sure what I described explains that part (naturally on the same vague level as OP itself). –  Marek Sep 7 '13 at 15:14

Why we need such property?

We do not need it, either we assume that it holds or we do not, depending on the phenomenon we seek to model.

Why is it essential and ubiquitous?

Define "essential". Ubiquitous: is it? Markovianity may be used in many situations because it makes computations possible and when the "physics" of a lot of phenomena make it plausible, from what we understand of said phenomena.

What is the "philosophy" of such property?

Define "philosophy". (And the answer is probably: "None".)

Is this property an assumption when modelling?

Yes, one assumes that it holds (or not) hence, when used, this is definitely an assumption.

If it is, does it hold necessarily?

Of course not (this seems to be a matter of pure logic, no?).

If it does not hold, what's the influence?

Define "influence" (and influence on what?).

share|improve this answer
    
This is my try to answer what I understand from the question (which is not much, since my opinion is that there is basically no question here). I only post it because the OP summoned me to do so. –  Did Sep 7 '13 at 13:33
    
Well, there's certainly no answer here, just nit-picking. The OP on the other hand, while vague, definitely contains a question. –  Marek Sep 7 '13 at 15:17
    
@Marek Thanks for your opinion about this answer. It might not have been crucial to post said opinion, but why not if it makes you feel better. –  Did Sep 7 '13 at 15:48
    
Yes, my question is vague because I do know little. Maybe I should give an example like weather etc. It may facilitate discussion. –  John Hass Sep 7 '13 at 23:39
    
@Joe Indeed, to make the question as specific as you can would be a big plus. Maybe write a new post based on a restricted example, asking a restricted question, and describing precisely the type of answer you seek. –  Did Sep 8 '13 at 8:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.