5
$\begingroup$

The Markovian assumption is used to model a number of different phenomena.

It basically says that the probability of a state is independent of its history, but only depends upon its immediately previous state.

That is,

$p(x_i|x_0,x_1, ..., x_{i-1}) = p(x_i|x_{i-1})$

Such a property has been used to model a number of phenomena, like the strength degradation of a structural component using dynamic bayesian networks.

I'm curious, what could be the limitations of such an assumption? Can you give some practical examples when such an assumption would be invalid? What could be the consequences in such a case?

$\endgroup$
3
  • $\begingroup$ Any system that has "memory" will not be Markov. e.g., you are not a Markov chain, as you presumably learn from the past, instead of just randomly moving one step from any present point. $\endgroup$
    – Jair Taylor
    Sep 12, 2019 at 5:10
  • $\begingroup$ @JairTaylor Sort of. If the memory has a finite bound—it only goes back at most $m$ steps for some fixed $m$—it might still be possible to model it with a Markov chain in which the states are $m$-tuples of the primitive states of the system. $\endgroup$
    – amd
    Sep 12, 2019 at 7:18
  • $\begingroup$ @amd Fair enough. Although the number of $m$-tuples grows exponentially with $m$, so things become difficult pretty quickly. $\endgroup$
    – Jair Taylor
    Sep 12, 2019 at 20:02

1 Answer 1

Reset to default
3
$\begingroup$

Any situation where a system has hidden state is going to be non-Markovian. For this, people invented hidden Markov models, where the system is assumed to be Markovian with a larger state but we can only observe part of it (see for instance the concrete example given here). Every system can be written as a hidden Markov model in a boring way: if the sequence is $X_1,X_2,\ldots$ then we can consider it as hidden Markov with states being sequences themselves $(X_1,\ldots,X_n)$ transitions to $(X_1,\ldots,X_{n+1})$ and the hidden Markov function maps a sequence to its final element.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.