3
$\begingroup$

Let $X_n$ be an positive recurrent, aperiodic Markov Chain and $T_0$ be the time of first return to state $0$. We learned in class that the invariant measure $\mathbf{y}$ of a Markov Chain can be written

$$y_i=\mathbf{E}\left[ \sum_{i=n}^\infty \mathbf{1}\{X_n=y_i\}\mathbf{1}\{n\leq T_0\}\right], \text{ and setting } y_0=1.$$

In other words, the entries of $\mathbf{y}$ are the number of visits to that state before returning to state $0$. This definition produces an invariant measure with entries $\in\mathbb{N}$. Since the stationary distribution $\mathbf{\pi}$ is just a normalized version of the unique invariant measure, does that mean that it's entries of $\mathbf{\pi}$ are always rational, even if the entries of the transition probability matrix aren't rational?

Since the stationary distribution of a Markov Chain is an eigenvector of the TPM, this would imply that matrices with transcendental components would always have a rational eigenvector, which seems... wrong.

$\endgroup$
  • $\begingroup$ WHy do you think that the sum of that series is a rational number? (See math.stackexchange.com/questions/116269/…) $\endgroup$ – Mariano Suárez-Álvarez Nov 11 '17 at 2:59
  • $\begingroup$ Each sum is an integer but the expectation need not be an integer or even rational. Recall that even a Bernoulli r.v. can have an expectation anywhere in [0,1]. $\endgroup$ – Ian Nov 11 '17 at 6:00
  • $\begingroup$ I think I was thrown off taking $T_0$ to be a fixed time and not a random variable itself. I admit I feel kinda silly reading this question in the morning. Clearly an infinite series, and thanks for clarifying. Just leaving this post up to share my gratitude, and then I'll delete it. $\endgroup$ – Jake Stevens-Haas Nov 11 '17 at 16:41
3
$\begingroup$

Consider the discrete markov chain with probability transition matrix $\begin{pmatrix} 1- 1/\pi & 1/\pi \\ 1 & 0 \end{pmatrix}$. The stationary distribution is $(\frac{\pi}{\pi+1}, \frac{1}{\pi+1})$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.