Let $Q$ be a $k\times k$ stochastic matrix. Let $A$ be a $k\times k$ matrix each of whose entries is $1/k$. For $0<p<1$, let


Show that $P$ is the transition matrix for an ergodic Markov chain.

I'm not really sure how to start on this one, but an ergodic chain means that you can go from any state, to any other state, that is, all entries should be $>0,$ if I'm not mistaken. Do I need to show that $P_{ij}>0 \ \forall \ i,j$ ?

EDIT: adding an attempt based on A.Pongracz comment.

Multiplying in $p$ and $1-p$ we have that

\begin{align}P&=\begin{pmatrix}pQ_{11} & ... &pQ_{1k}\\\vdots & \ddots & \vdots \\ pQ_{k1} & \dots & pQ_{kk}\end{pmatrix}+\begin{pmatrix}\frac{1-p}{k} & ... &\frac{1-p}{k}\\\vdots & \ddots & \vdots \\ \frac{1-p}{k} & \dots & \frac{1-p}{k}\end{pmatrix}\\ &=\begin{pmatrix}pQ_{11}+\frac{(1-p)}{k} & ... &pQ_{1k}+\frac{(1-p)}{k}\\\vdots & \ddots & \vdots \\ pQ_{k1}+\frac{(1-p)}{k} & \dots & pQ_{kk}+\frac{(1-p)}{k}\end{pmatrix}.\end{align}

We know that $\frac{1-p}{k}>0$, and since every $Q_{ij}>0$ all the entries of $P$ must be strictly positive.

Adding $\frac{1-p}{k}$, $k$ number of times we get for the first row row that

$$p(\underbrace{Q_{11}+Q_{12}+...+Q_{k1}}_{=1 \ \text{cuz Q is stochastic}})+1-p=p+1-p=1.$$

similar calculations can be applied to all rows and we have shown that $P$ is a transition matrix for an ergodic Markov chain.


It is SUFFICIENT for a square matrix to be the transition matrix of an ergodic chain that all its entries be strictly positive. (There are conflicting terminologies, but this condition is surely stronger.) So yes, that is a good way to go, but also, don't forget to check that $P$ is also stochastic. That is, the sum of each row is $1$.

  • $\begingroup$ Thanks, I'll make an attempt and edit it in my answer! $\endgroup$ – Parseval Jan 2 at 18:38
  • $\begingroup$ Please see my edit. Why doesn't that method work? and how can I check that the sum of those rows is 1 when I don't know the entries? $\endgroup$ – Parseval Jan 2 at 20:07
  • $\begingroup$ I think I got it now! Can you plese check if that is correct? $\endgroup$ – Parseval Jan 2 at 20:44
  • $\begingroup$ Yes, it looks good. $\endgroup$ – A. Pongrácz Jan 2 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.