6
$\begingroup$

Why is the process $Q(t)$ defined below a Markov process ? $$Q(t)=Q(0)+A(t)- S\left(\int_0^t Q(s)\,\mathrm ds\right)$$

where $A$ and $S$ are unit rate Poisson process.
Since the integral depends on the past, shouldn't this be non-Markov ? What is the intuition for this ?

$\endgroup$
2
  • $\begingroup$ Why do you think it is a Markov process? The pathwise definition of $Q$ is by means of an integral equation. $\endgroup$
    – William M.
    Jan 10, 2023 at 19:43
  • $\begingroup$ The paper I am reading says it's Markov $\endgroup$
    – abc
    Jan 10, 2023 at 20:01

1 Answer 1

4
$\begingroup$

The Markov property means that the evolution of a stochastic process doesn't depend on the past, but the current state can be a function of past values (at least of the initial state).

In classical calculus, it would be translated by the fact that the derivative of the considered quantity at time $t$ is independent of past values; however, as stochastic processes are not differentiable, we are constrained to study their increments instead.

In the present case, the variation of the stochastic process $Q$ over a time interval $[s,t]$ is given by $$ Q(t) - Q(s) = A(t) - A(s) - \left(S\left(\int_0^tQ(\tau)\mathrm{d}\tau\right) - S\left(\int_0^sQ(\tau)\mathrm{d}\tau\right)\right), $$ with $A(t) \sim S(t) \sim \mathcal{Poisson}(t)$, hence $A(t)-A(s) \sim \mathcal{Poisson}(t-s)$ and $S\left(\int_0^tQ(\tau)\mathrm{d}\tau\right) - S\left(\int_0^sQ(\tau)\mathrm{d}\tau\right) \sim \mathcal{Poisson}\left(\int_s^tQ(\tau)\mathrm{d}\tau\right)$, such that in fine $$ Q(t)-Q(s) \sim \mathcal{Poisson}\left(t-s - \int_s^tQ(\tau)\mathrm{d}\tau\right). $$ This expression doesn't contain times prior to $s$, that is why the increments of $Q$ only depend on the "present" values over the interval $[s,t]$, hence memorylessness / Markov property.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .