# Continuous time Markov Chain help

Suppose that the price of company stock is increasing if the latest news are positive. News article arrive with a Poisson distribution of intensity 2 per day. Furthermore , good news arrive with probability of 2/3.

Find the Q matrix and the long term proportion of time that the price of a stocks is increasing.

This is a practice HW. I dont know how solve continuous time markov chain. What we have seen in class is the Poisson process , the M/M/1 and the Birth and death model. I'm not sure which model to apply.

• You first determine the number of states (here the number of states is finite and small). Then you determine the transition rates between states. This is simpler than an M/M/1 queue and is one of the simplest possible birth-death chains. Commented Jun 23, 2017 at 20:32
• there's only 2 state , so Q= { { -2/3 , 2/3 } , {1/3 ,-1/3 } } Commented Jun 23, 2017 at 20:52
• So, what is the physical meaning of your states, and how did you get $q_{12} = 2/3$? [In other words, how does one verify your work?] Commented Jun 23, 2017 at 21:01
• its the probability of good news $q_{12}=2/3$ Commented Jun 23, 2017 at 21:04
• The above $q_{21}$ and $q_{12}$ values might be mixed up, depending on the definitions of states, for 2=bad, 1=good I think it should be the reverse of your above comment. Commented Jun 23, 2017 at 22:03

This answer summarizes our discussion. We have a continuous time Markov chain with 2 states \begin{align} \mbox{state 1} &= \mbox{good news} \\ \mbox{state 2} &= \mbox{bad news} \end{align} The $2 \times 2$ transition rate matrix $Q$ has important parts given by:
$$Q = \begin{bmatrix}* & q_{12} \\ q_{21} & * \end{bmatrix}$$ where $q_{ij}$ is the transition rate associated with going from $i$ to $j$: \begin{align} q_{12} &= \mbox{rate of getting bad news, given we are in state 1} \\ q_{21} &= \mbox{rate of of getting good news, given we are in state 2} \end{align} Suppose we are in the bad state (state 2). Then transition rates are described by the second row of $Q$ and: $$\underbrace{\mbox{rate of good news}}_{q_{21}} + \mbox{rate of bad news} = 2 \: (\mbox{per day})$$ We can derive the value of $q_{21}$ by viewing this as the i.i.d. split of a Poisson process of rate $2$, and using results of splitting Poisson processes.
Once we have the $Q$ matrix, we can get the steady state values for each state by solving the corresponding birth-death chain. You will find that the steady state will only depend on the splitting probability and not on the rate $\lambda$ of the Poisson process. On the other hand, the $Q$ matrix will be proportional to $\lambda$ (in this case, $\lambda= 2$ per day).