# How to solve a SDE defined via a Markov Process?

I have to solve the following SDE. $$\mathrm dY_t= f(X_t) \mathrm dt, \tag{1}$$ where $X_t$ is a two-state Markov Process possesses states $a$ and $b$.

Moreover, I would like to solve $$\mathrm dY_t= Z_t \mathrm dt, \tag{2}$$ where $Z_t$ is an rv conditioned on $X_t$, i.e., $Z_t \mid X_t \sim Bin(n,g(X_t))$.

After seeing Did's comment and Jay.H's answer, I tried to derive $Y_t$ with Riemann Sum. Let us introduce $$Y_t := \lim_{\lambda \rightarrow 0} \sum_{t_i} f(X_{t_i}) \Delta t,$$ where $0=t_0<t_1<t_2<\cdots<t_n<t_{n+1}=t$, $\lambda=\max(t_{j+1}-t_j)$ for $j=0 \ldots n$. Assume that the Markov process has a stationary state $\pi$, and $\pi_a$, $\pi_b$ are probabilities of the state $a$ and $b$ respectively. Since the terms added are infinite, thus $$\Pr \{ \lim_{\lambda \rightarrow 0} \sum_{t_i} f(X_{t_i}) \Delta t\ =\pi_a f(X_a) + \pi_b f(X_b)\} = 1,$$ namely $$\Pr \{ Y_t =\pi_a f(X_a) + \pi_b f(X_b)\} = 1.$$ Therefore, we can define $Y_t:=\pi_a f(X_a) + \pi_b f(X_b)$ with probability 1.

It's very strange that $Y_t$ is a deterministic real number, what's wrong with the above derivation?

• How does one define $dX_t$ when $X$ is a (presumably, real-valued) two-state process? – Did Jan 6 '16 at 9:27
• Sorry for the typo, it's $\mathrm dt$ instead of $\mathrm dX_t$.@Did – robit Jan 6 '16 at 9:51
• What exactly do you mean by "solve"? What kind of result do you expect? – saz Jan 6 '16 at 13:04
• Not an SDE anymore, then. A (strong) solution of the (new) equation is simply $$Y_t(\omega)=Y_0(\omega)+\int_0^tf(X_s(\omega))\,ds=Y_0(\omega)+tf(a)+{}{}{}{}{}{}L_t^b(\omega)(f(b)-f(a)),$$ where $$L_t^b(\omega)=\int_0^t\mathbf 1_{X_s(\omega)=b}\,ds.$$ – Did Jan 6 '16 at 14:57
• I would like to obtain an explicit formula of $Y_t$. @saz – robit Jan 7 '16 at 3:24

Let $a$,$b$ be the two states of $X_t$, and let $T_{a,t}$ be the time that $X_s$ spend in state $a$, for $s\in [0\ t]$. The answer for (1) is:
$Y_t = f(a)T_{a,t} + f(b)(t-T_{a,t})$
For (2), I'm not sure the question is even well defined, it seems that each $Z_t$ has some "randomness" which is totally unrelated to each other, as a result, a sample path $Z_.$ may not be measurable in the usual sense (for example, Borel or Lebesgue)