There are some discrete-time stochastic systems with some input that have been considered. For example

$X_{n+1}=F(X_n, U_n)$

where $\{X_n\}$ is some stochastic process, let's say the Markov chain on $\mathbb R$ and $U_n$ is a sequence of inputs in $\mathbb R$, where $F$ is a non-linear function. Could anyone give me some references about what would be a continuous-time version of the system? Would it be

$\dot X_t=F(X_t,U_t)$,

where $X_t$ is a continuous time Markov chain on $\mathbb R$? If yes, how to understand the $\dot X_t= \frac{d X_t}{dt}$ part? Can we easily define derivative of a continuous time Markov chain? Thanks for any help.

  • $\begingroup$ Are you familiar with Ito calculus and Ito's lemma? $\endgroup$ Commented Mar 22, 2022 at 21:24
  • 1
    $\begingroup$ Yes, I did a course on stochastic calculus long ago, could you be more elaborate, please? $\endgroup$
    – Myshkin
    Commented Mar 22, 2022 at 23:07
  • $\begingroup$ Yeah, sorry, that wasn't meant to be rude. My comment was getting too long so I posted it as an answer, but I'm not sure if it quite answers your question. $\endgroup$ Commented Mar 23, 2022 at 13:31

1 Answer 1


Most continuous stochastic processes are not differentiable, and in fact are nowhere differentiable. While we cannot define the derivative of most continuous stochastic processes, we can define an integral with respect to some of them. The question of what the continuous-time version of a system is depends on the particulars of the system. For one example, we could have something like $$dX_t = F(X_t,U_t)dt + G(X_t,U_t)dW_t$$ where $W$ is a Brownian motion. The notation is a little confusing since, while this does describe how $X_t$ changes over time, it's shorthand for an integral rather than a derivative. This example would mean $X_t = X_0 + \int_0^t F(X_s,U_s)ds + \int_0^t G(X_s,U_s)dW_s$.

For references, you might look at Oksendal's Stochastic Differential Equations, Revuz and Yor's Continuous Martingales and Brownian Motion, or Karatzas and Shreve's Brownian Motion and Stochastic Calculus.

  • $\begingroup$ Basically, the short answer is "No, we can't define the derivative of a continuous stochastic process." The closest analogue to something like $X_{n+1} = F(X_n,U_n)$ is a stochastic differential equation, which describes how a stochastic process evolves over time. $\endgroup$ Commented Mar 24, 2022 at 14:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .