Right so I found these questions at the end of our chapter on discrete markov chains. The question asks whether they are Markov chains. I can see some being continuous markov chains and wonder if the question means to ask determine which are discrete markov chains. I provide what I think they are. Please guide me if I'm wrong.

A) process $ X_n $ first arrives at state i and then stays there for some 2+N steps and then moves to state i+1. N is a random variable as value 1 with probability 0.5 and value 2 with probability 0.5.

My answer : continuous markov chain. Thus because the probability of reaching the next state still remains the same regardless of the length of the time step. Thus the next state only depends on its current state. Please correct me where I might have gone wrong here. Or is it not a markov chain because the transition probability depends its current state, current place in time and the length of the interval.

B) $X_n =An$ where A is a random variable with - 1 with probability 0.5 and 1 with probability 0.5.

My answer : not a markov chain of any kind as the probability of the next state does not depend on the current state.

C) if $X_n = i$ then $X_{n+1} =i$ with probability 0.5. Then $X_{n+1} = i+1$ with probability 0.5. However if the process stays longer than 3 steps in $i$ then it moves to $i+1$.

My answer : the probability of reaching each state does depend on the time interval as an additional condition. This means the time interval changes the transition probability. Thus not a markov chain.

  • 1
    $\begingroup$ You might want to choose a definition of Markov chain to be able to prove something is a Markov chain. $\endgroup$ – Ton May 29 '18 at 22:05
  • $\begingroup$ Are you sure about your answer in A? What is the definition of a markov chain? It is that $\mathbb{P}(X_n = j \vert X_{n-1} = i_{n-1}, \dots X_0 = i_0) = \mathbb{P}(X_n = j \vert X_{n-1} = i_{n-1})$. In other words, a markov chain is memoryless. Let's say you're at position $i$ right now at $X_n$, and you know that at $X_{n-1}$ and $X_{n-2}$ you were also at position $i$. Is this equivalent (w.r.t to transition probability) to $\textbf{just}$ knowing you were at position $i$ only just before? $\endgroup$ – blanchey May 29 '18 at 22:44
  • $\begingroup$ Also, I believe that B is a markov chain. Can you use the definition I provided you with? Just because the distribution of the next state does not depend on the current one does not imply that it is not a markov chain. If, in fact, the distribution of the current state is irrespective of any of the previous states, it is a markov chain. $\endgroup$ – blanchey May 29 '18 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.