2
$\begingroup$

In my linear algebra class, we were learning about Markov chains, and the prof. put a confusing question on the board. Namely, can one have a transition matrix in which each row is the same, i.e., let's say for a $3 \times 3$ matrix, each row is $[a,b,c]$, with the constraint being that $a + b +c =1$. Is this possible? Further, what if the values of $a,b,c$ are the same as the values of the steady-state vector?

$\endgroup$
  • 2
    $\begingroup$ Why not? It might be a peculiar Markov chain, but the matrix fits all the conditions. $\endgroup$ – kimchi lover Aug 10 '17 at 2:47
2
$\begingroup$

The values $a,b,c$ are going to be the values of the steady state vector - we can think about this as such: in the long run, the probability from going from any state to state 1 is $a$, and correspondingly $b$ and $c$ for states 2 and 3 respectively.

The transition matrix you mentioned is definitely a valid transition matrix. The only requirement is that each row sums to 1.

$\endgroup$
2
$\begingroup$

A Markov chain $X_1, X_2, \dots$ with a transition matrix in which each row is the same corresponds to the special case in which each $X_i$ is independent of $X_{i-1}.$

In general, a Markov chain has $$P(X_i = x_i|X_{i-1}=x_{i-1},\, X_{i-2} = x_{i-2},\, \dots,\, X_{i-k} = x_{i-l}) = P(X_i = x_i|X_{i-1}=x_{i-1}),$$ so colloquially, dependence is only on the most recent specified step. However, there is no requirement for any dependence at all. An sequence of independent random variables can be Markovian.

Note: The power $P^n$ of a Markov chain shows $n$-step transition probabilities. Some Markov chains have the property that $P^n,$ for sufficiently large $n,$ has rows that are very nearly the same, indicating that $\lim_{n\rightarrow\infty} P^n$ exists. This means that, with sufficient passage of time, the state of a Markov chain at step $n$ does not depend on its initial state at step $1.$ A chain with such limiting distribution is called ergodic.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.