4
$\begingroup$

If I am not mistaken, the steady state distribution is independent of initial state distribution, and regular Markov chains satisfies this definition.

On the other hand, since the row of each limiting matrix for an absorbing Markov chain is the same, the state distribution after a large number of transitions for an absorbing Markov chain is dependent on the initial state distribution.

However, I read somewhere that an absorbing Markov chain can have steady state distributions - which contradicts what I have always believed. I mean, I know what they are trying to imply (as in the chain will converge to a fixed probability), but is it the correct term?

$\endgroup$
3
  • $\begingroup$ can you write everything is term of matrix/vectors (and eigenvectors) ? $\endgroup$
    – reuns
    Jul 9, 2016 at 17:10
  • 1
    $\begingroup$ Every distribution whose support is included in the set of absorbing states is a steady state distribution, by definition. Steady state distributions need not be independent of the initial distribution, nor are they always unique. $\endgroup$
    – Did
    Jul 9, 2016 at 17:14
  • $\begingroup$ @user1952009 Indeed, my statement was based on a faulty assumption. Retracted. $\endgroup$
    – Math1000
    Jul 9, 2016 at 17:50

2 Answers 2

1
$\begingroup$

Let $X=\{X_n:n\in\mathbb N_0\}$ be a Markov chain with state space $S$ and transition probabilities $p_{ij}$. A vector $\pi\in\mathbb R^{\#S}$ is a stationary distribution for $X$ iff

  1. $\pi_j\geqslant0$ for all $j\in S$,
  2. $\pi_j = \sum_{i\in S}\pi_j p_{ij}$ for all $j\in S$,
  3. $\sum_{j\in S} \pi_j = 1$.

For each state $i\in S$, let $C_i = \{j\in S : i\leftrightarrow j\}$ be the communicating class of $i$. Assume that each class is positive recurrent, that is, for each $i\in S$, the return time $$\tau_{ii} = \inf\{n>0: X_n = i\mid X_0=i\} $$ has finite expectation. Then there exists a unique stationary distribution $\pi^{(i)}$ for $X$ conditioned on $X_0\in C_i$, with $$\pi^{(i)}_j = \frac1{\mathbb E[\tau_{jj}]},\ j\in C_i. $$ Let $\bar\pi^{(i)}$ be the extension of $\pi^{(i)}$ to $S$ with $\bar\pi^{(i)}_j=0$ for $j\in S\setminus C_i$, then the set of stationary distributions for $X$ is the convex hull of $\{\bar\pi^{(i)}:i\in S\}$. If there is a class $C_i$ which is not positive recurrent, then no stationary distribution exists for that class, and hence no stationary distribution exists for $X$.

In the case where $X$ is absorbing, however, the communicating classes are simply the singletons $\{i\}$ for the absorbing states with $p_{ii}=1$. Let $A\subset S$ be the set of absorbing states, then $\pi$ is a stationary distribution iff $\pi_j=0$ for $j\in S\setminus A$.

$\endgroup$
5
  • $\begingroup$ Apparently this contradicts what Did has said. May I know how you can counter his argument or is his argument compatible with yours? $\endgroup$
    – Michael
    Jul 10, 2016 at 4:45
  • $\begingroup$ @Michael How so? An absorbing state is a class with one state... $\endgroup$
    – Math1000
    Jul 10, 2016 at 4:50
  • 1
    $\begingroup$ There is no contradiction and the relevance of what is explained in this answer is minimal since, having absorbing states drastically reduces the positively recurrent classes (basically there is none except each absorbing state on its own)... $\endgroup$
    – Did
    Jul 10, 2016 at 7:41
  • $\begingroup$ @Did Fair enough, I will add that to the answer. $\endgroup$
    – Math1000
    Jul 10, 2016 at 7:43
  • $\begingroup$ 1. Is $C_i communicating class value binary? So 1 if here is i <-> j and 0 otherwise? 2. Is E[T_jj] in 1/E[T_jj] an Eigen matrix, and if so how do I compute it based on T_jj? $\endgroup$
    – Csaba Toth
    Feb 9 at 23:18
0
$\begingroup$

I believe there are two conceptions are related to your question.

  1. The first conception is absorbing probabilities. It gives us the information about how this absorbing Markov chain ends. But the final distribution still depends on initial distribution.

  2. Quazi stationary distribution. It is independent with the initial distribution but only works when absorbing Markov chain is long and it is not absorbed.

Hope this can help you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.