6
$\begingroup$

I know that, for a Markov Chain, a stationary distribution is the (row) vector $\pi$ such that $\pi \cdot P = \pi$, where $P$ is the one-step transition matrix for the MC. Intuitively, I assume that this means that $\pi$ is the collection of probabilities that the process ends up in each of the possible states in the long run.

My understanding of equilibrium distributions is that the (row) vector $P^{*} = (P_{1}^{*}, P_{2}^{*}, \dots, P_{m}^{*})$ is an equilibrium distribution if each $P_{j}^{*} = \lim_{n \rightarrow \infty} P_{ij}^{(n)}$ exists for every $j \in S$ and is not dependent of the initial state $i$. This implies that $P^{*}$ is the collection of probabilities of the process being in each state after $\infty$ steps.

Does this not mean that equilibrium distributions are the same as stationary distributions, in practice?

$\endgroup$
0

1 Answer 1

4
$\begingroup$

"Does this not mean that equilibrium distributions are the same as stationary distributions, in practice?"

Stationary Distributions:

Let $\mathbf{P}$ be the transition probability matrix of a homogeneous Markov chain $\{X_n, n \geq 0\}$. If there exists
a probability vector $\mathbf{\pi}$ such that $$\mathbf{\pi} \mathbf{P} = \mathbf{\pi} \:\:\:\:\:\:\: (1)$$

then $\mathbf{\pi}$ is called a stationary distribution for the Markov chain.

Equilibrium Distributions:

Thm:

Let $\{X_n, n \geq 0\}$ be a regular homogeneous finite-state Markov chain

with transition matrix $\mathbf{P}$.

Then $$\lim \limits_{n \to \infty} \mathbf{P}^n = \mathbf{\hat{P}}$$ where $\mathbf{\hat{P}}$ is a matrix whose rows are identical and equal to the stationary distribution $\mathbf{\pi}$ for the Markov chain defined by $(1)$.


To express the relationship in another way:

Let $\mathbf{\pi}$ be the stationary distribution. $\mathbf{\pi}$ is related to the expected return time $\mu_j$ by:

$$\mathbf{\pi}_j = \frac{1}{\mu_j}$$

If the chain is both irreducible and aperiodic, it is said to have an equilibrium distribution

$$\lim_{n \to \infty} p_{ij}^n = \frac{1}{\mu_j}$$

Such $\mathbf{\pi}$ is called the equilibrium distribution.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .