Fix an aperiodic discrete-time Markov chain $X_0,X_1,\dots$, with $\mathbb P_i$ denoting probabilities according to $X_0=i$. Define the return probability $$p_{ii}^{(n)} = \mathbb P_i[X_n=i].$$

Is it true that $$\sum_{n=0}^\infty |p_{ii}^{(n)}-p_{ii}^{(n+1)}|<\infty\text{?}$$

I am most interested in the null-recurrent case. For transient states there is a positive answer: the return probabilities are themselves summable. I know that for finite state spaces there is exponential convergence to some stationary state, which also implies a positive answer.

  • $\begingroup$ Did you try to compute $p^{(n)}_{ii}$ for the symmetric random walk on $\mathbb Z$ with transitions $i\to i$ and $i\to i\pm1$, for every $i$ in $\mathbb Z$? $\endgroup$ – Did Sep 13 '17 at 17:37
  • $\begingroup$ @Did: when the self-transition probability is $1/2$ and $i\to i\pm 1$ probabilities each $1/4$, the probabilities are $2^{-2n}\binom{2n}{n}$ which is non-increasing so the differences sum to at most $1$. I don't know about other other self-transition parameters but my guess is they'd behave similarly at least in the long run. $\endgroup$ – Dap Sep 13 '17 at 17:48
  • $\begingroup$ Right, random walks look like a dead end. Sorry for the noise. $\endgroup$ – Did Sep 13 '17 at 19:37

The answer is no, as shown in the paper "On the problem by Erdős-de Bruijn-Kingman on regularity of reciprocals for exponential series" by Alexander Gomilko, Yuri Tomilov, https://arxiv.org/abs/1701.04357


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.