3
$\begingroup$

Consider two irreducible ergodic Markov chains with the same state space $\{0, 1, . . . , N\}$, with transition matrices $P$ and $Q$ and respective stationary distributions $\pi$ and $\rho$. We define two processes, each with initial state 0, as follows. For the process $(X_n)$, a fair coin is flipped and if it comes up Heads then all the following states are obtained using the transition matrix $P$, whereas if it comes up Tails then all the following states are obtained using the transition matrix $Q$. For the process $(Y_n)$, the coin is flipped at each stage, and when it is Heads then the next state is chosen using $P$, whereas when it is Tails then the next state is chosen using $Q$.

a. Is the process $(X_n)$ a Markov chain? Is the process $(Y_n)$ a Markov chain? Explain your reasoning. If one or both is a Markov chain, what are the transition probabilities?

Attempt:

$(X_n)$ is not a MC, since the next state transition probability depends on the initial state (which you flip the coin).

$(Y_n)$ is a MC, because the next state transition only depends on the current state. Also, stationarity holds in $Y_n$. The transition probability should be $1/2(P_{i,j}+Q_{i,j}) $

------------------------

b. Determine the $\lim_{n \rightarrow \infty}P(X_n=j)$ for each state j.

Attempt:

This would be $1/2(\pi_j+\rho_j)$

$\endgroup$
  • $\begingroup$ i believe your solutions are right - though some justification is lacking. $\endgroup$ – leonbloy Apr 11 '16 at 0:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.