0
$\begingroup$

I have very recently started learning about Markov chains. I know what the Stochastic matrix is. However, I came across a question like:

If a transition probability matrix is of order $n\times n$ then number of steady state equations would be:

  1. $n$
  2. $n^2$
  3. $n-1$
  4. $n+1$

I'm not sure what they mean by "steady state equations". Haven't come across that term while learning Markov chains. Could someone please explain or provide some reference?

$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

It's a bit unclear out of context, but I'd expect the equations to be $\pi P=\pi$, where $\pi$ is a row vector, along with the normalization requirement $\pi 1=1$ where $1$ is a column vector of all ones. That would be $n+1$ linear equations. If you don't have a normalization requirement, then $n$ would also be a sensible answer.

$\endgroup$
4
  • $\begingroup$ It's $\pi^\top 1 = 1$. That's automatic for a stochastic matrix. But note that the rank of $\pi-I$ is $n-1$ (generically), so shouldn't the answer be $n-1$? (I don't think the normalization requirement $1^\top P = 1$ should count, but maybe it should.) $\endgroup$
    – Ted Shifrin
    May 25, 2018 at 2:35
  • $\begingroup$ @TedShifrin You still get $n$ equations, but only $n-1$ of them are independent. Depends on how you’re counting, I suppose. $\endgroup$
    – amd
    May 25, 2018 at 2:39
  • $\begingroup$ @TedShifrin $\pi$ is a row vector so there is no $T$ required. The normalization requirement is also not automatic, there are non-normalized invariant measures. That said, it does indeed depend on your counting convention, because the rank of $\pi-I$ is necessarily at most $n-1$, so there must be a redundant equation. But you can't unambiguously answer the question if you adopt that perspective. $\endgroup$
    – Ian
    May 25, 2018 at 3:06
  • $\begingroup$ I use column vectors, so I swapped the meanings of $\pi$ and $P$. Sorry. Yes, terrible question all around. $\endgroup$
    – Ted Shifrin
    May 25, 2018 at 3:08

You must log in to answer this question.