3
$\begingroup$

If I am not mistaken, the steady state distribution is independent of initial state distribution, and regular Markov chains satisfies this definition.

On the other hand, since the row of each limiting matrix for an absorbing Markov chain is the same, the state distribution after a large number of transitions for an absorbing Markov chain is dependent on the initial state distribution.

However, I read somewhere that an absorbing Markov chain can have steady state distributions - which contradicts what I have always believed. I mean, I know what they are trying to imply (as in the chain will converge to a fixed probability), but is it the correct term?

$\endgroup$
  • $\begingroup$ can you write everything is term of matrix/vectors (and eigenvectors) ? $\endgroup$ – reuns Jul 9 '16 at 17:10
  • 1
    $\begingroup$ Every distribution whose support is included in the set of absorbing states is a steady state distribution, by definition. Steady state distributions need not be independent of the initial distribution, nor are they always unique. $\endgroup$ – Did Jul 9 '16 at 17:14
  • $\begingroup$ @user1952009 Indeed, my statement was based on a faulty assumption. Retracted. $\endgroup$ – Math1000 Jul 9 '16 at 17:50
1
$\begingroup$

Let $X=\{X_n:n\in\mathbb N_0\}$ be a Markov chain with state space $S$ and transition probabilities $p_{ij}$. A vector $\pi\in\mathbb R^{\#S}$ is a stationary distribution for $X$ iff

  1. $\pi_j\geqslant0$ for all $j\in S$,
  2. $\pi_j = \sum_{i\in S}\pi_j p_{ij}$ for all $j\in S$,
  3. $\sum_{j\in S} \pi_j = 1$.

For each state $i\in S$, let $C_i = \{j\in S : i\leftrightarrow j\}$ be the communicating class of $i$. Assume that each class is positive recurrent, that is, for each $i\in S$, the return time $$\tau_{ii} = \inf\{n>0: X_n = i\mid X_0=i\} $$ has finite expectation. Then there exists a unique stationary distribution $\pi^{(i)}$ for $X$ conditioned on $X_0\in C_i$, with $$\pi^{(i)}_j = \frac1{\mathbb E[\tau_{jj}]},\ j\in C_i. $$ Let $\bar\pi^{(i)}$ be the extension of $\pi^{(i)}$ to $S$ with $\bar\pi^{(i)}_j=0$ for $j\in S\setminus C_i$, then the set of stationary distributions for $X$ is the convex hull of $\{\bar\pi^{(i)}:i\in S\}$. If there is a class $C_i$ which is not positive recurrent, then no stationary distribution exists for that class, and hence no stationary distribution exists for $X$.

In the case where $X$ is absorbing, however, the communicating classes are simply the singletons $\{i\}$ for the absorbing states with $p_{ii}=1$. Let $A\subset S$ be the set of absorbing states, then $\pi$ is a stationary distribution iff $\pi_j=0$ for $j\in S\setminus A$.

$\endgroup$
  • $\begingroup$ Apparently this contradicts what Did has said. May I know how you can counter his argument or is his argument compatible with yours? $\endgroup$ – Michael Jul 10 '16 at 4:45
  • $\begingroup$ @Michael How so? An absorbing state is a class with one state... $\endgroup$ – Math1000 Jul 10 '16 at 4:50
  • 1
    $\begingroup$ There is no contradiction and the relevance of what is explained in this answer is minimal since, having absorbing states drastically reduces the positively recurrent classes (basically there is none except each absorbing state on its own)... $\endgroup$ – Did Jul 10 '16 at 7:41
  • $\begingroup$ @Did Fair enough, I will add that to the answer. $\endgroup$ – Math1000 Jul 10 '16 at 7:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.