7
$\begingroup$

I was trying to understand the difference between strong Markov property and the usual Markov property for a discrete number of states. I think I understand why the strong Markov property implies the usual one : We have to consider a deterministic stopping time $T(\omega)=t_0$, right?
But what would be a simple example (like coin toss, dice toss,...) where we have the Markov property but not the strong Markov property?

$\endgroup$

1 Answer 1

6
$\begingroup$

Any discrete time Markov chain has the strong Markov property; the same holds for a continuous time chain with a stopping time that only takes a countable set of values. This is actually not too bad to prove, just conditioning on the particular realisation of the stopping time. You can read about the strong Markov property in James Norris' book, in particular Section 1.4 -- this is for discrete time. As you point out, strong Markov implies Markov by just taking the stopping time deterministic. So you can't come up with a chain that has one but not the other :)

For continuous time, it is more subtle. These lecture notes give a counterexample -- Example 167. (The full lecture notes can be found here.)

$\endgroup$
2
  • $\begingroup$ Could you clarify why the counterexample (Example 167) cannot be converted to the discrete-time setting? I can't quite put my finger on it. $\endgroup$ Commented Dec 26, 2022 at 1:28
  • 1
    $\begingroup$ @paperskilltrees I haven't thought a great deal about it, but I think that one reason is because of the need to have lack of invertibility at precisely one point, which has 'mass' 0. This rules out a discrete-space counter-example. Of course, one can have discete-time + continuous-space. I expect that this will have the issue that hitting particular locations will have probability 0. I'm not sure beyond this, though. Continuous-space stuff is not my area of expertise! You could ask a new question, though. If you do, please link here! $\endgroup$
    – Sam OT
    Commented Jan 3, 2023 at 8:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .