0
$\begingroup$

I have the following equation where $P$ is the tpm of an irreducible markov chain, and $0<p<1$.

\begin{align} \tilde{P}= pP+(1-p)I \end{align}

I want to show that $\tilde{P}$ is ergodic, which in turn means that I can also that $\tilde{P}$ is irreducible and aperiodic.

Intuitively, I think I can show that $\tilde{P}$ is aperiodic since its diagonal components can never have a zero value (because of the $(1-p)I$ term). However, I don't see how I can infer from this that $\tilde{P}$ is irreducible.

Does irreducibility of the Markov chains carry over when you are performing linear operations on its transition probability matrix?

I'm not sure if my intuition is perfectly correct either here, any help would be great!

$\endgroup$
1
$\begingroup$

One way to think about irreducibility is in graph-theory terms: a Markov chain is irreducible if, between any two states $x$ and $y$, there is a path the Markov chain could take to get from $x$ to $y$: a path along which the transition probabilities are all positive.

The transformed matrix $\tilde{P} = pP + (1-p)I$ preserves the positive entries of $P$: if $P_{ij} > 0$, then $\tilde{P}_{ij} = p P_{ij} + (1-p)I_{ij} \ge p P_{ij} > 0$.

As a result, if we had a path from state $x$ to state $y$ such that all the transition probabilities were positive in $P$, those transition probabilities stay positive in $\tilde{P}$, and so it remains a valid path showing we can get from $x$ to $y$ in the transformed Markov chain.

So if we started with an irreducible Markov chain, the result will still be irreducible.

And more generally, you ask:

Does irreducibility of the Markov chains carry over when you are performing linear operations on its transition probability matrix?

Well, to get a Markov chain out of linear operations on transition matrices, we must take a convex combination: some sum of the form $$ \lambda_1 P_1 + \lambda_2 P_2 + \dots + \lambda_n P_n $$ where $P_1, \dots, P_n$ are transition matrices, and $\lambda_1, \dots, \lambda_n$ are nonnegative real number weights with $\lambda_1 + \lambda_2 + \dots + \lambda_n = 1$. (This condition is necessary to make sure the result is still a transition matrix.)

The same argument as above tells us that if in this sum there is a single irreducible transition matrix $P_i$ with a positive coefficient $\lambda_i$, then the result will be irreducible.

(In general, though, the result could still be irreducible even if we started with non-irreducible matrices, just by accident.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.