Consider discrete stochastic process on some statespace $\mathbb X$. And some evolution $p(x_k\mid x_{k-1},x_{k-2},...)$, by defining $P_k=\{n\in\mathbb Z\mid n<k\}$ and taking as new statespace all functions: $$\mathcal C = \{f: P_k\to\mathbb X\mid k\in\mathbb Z\}$$

We can define a Markov process $p_{\mathcal C}(f_t\mid f_{t-1})$ (for $f_{t-1}:P_k\to\mathbb X$) on this statespace via: $$p_{\mathcal C}(f_t \mid f_{t-1}) =\begin{cases} \qquad 0 & f_t\text{ is not a map }P_{k+1}\to\mathbb X\\ \qquad 0&f_t(n)\neq f_{t-1}(n) \text{ for }n<k\\ p\left(\,f_t(k)\mid f_{t-1}(k-1),f_{t-1}(k-2),...\right)&\text{otherwise} \end{cases}$$

The idea is that basically every process is Markov if one considers it as acting on the statespace of histories. Doesn't seem like much of a problem to extend to continuous processes.

My question is, is this construction right or am I screwing something up here? I don't know much about stochastic processes but I think there exist theorems that hold only for Markov processes, this suggests though that they should hold for every process.


It is true that the new process you've constructed is Markov; let's call it $g(t)$. If you know $g(t-1) = f_{t-1} : \{1 , \dots, k-1 \} \to \mathbb{X}$ then you may calculate the probability of transitioning to $f_t$ where $f_t(k) = x$ so that $$ P(g(t) = f_t | g(t-1) = f_{t-1}) = P(f_t(k) = x | f_t(k-1), f_t(k-2), \dots). $$ However, this is a new process, and it evolves on a completely different state space than your original process, so it is unlikely that the Markovian nature of the process $g$ can be used to gain any understanding of the process $f$. In particular, when you say 'basically every process is Markov if one considers it as acting on the state space of histories' you are glossing over this leap we've taken from looking at $f$ (governed by $p$) to looking at $g$ (governed by $p_{\mathcal{C}}$).

If I can paraphrase you and repeat myself:

"The transition probabilities of a non-Markov process depend on the past as well as the present state. I'll define a process where the entire history is included in the information of 'present state' (i.e. work on the path space). This new thing must now be Markov."

This is true, but tautological, so it doesn't appear to garner us anything useful. If the original process were complex, the transition probabilities for $g$ are just as complex.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.