I'm a bit of a newbie when it comes to probability theory/terminology, so please excuse any errors on my part. Note that I ask this out of curiosity, so a full answer would be appreciated. Here goes the problem:

Define the probabilistic map $M:\Bbb N\mapsto\Bbb N$ as follows:


with $p_1$, ..., $p_6$ constant and $p_1+\cdots+p_6=1$. Now, define $t$ as the first value such that $M^t(1)=0$ (with $M^t$ denoting repetition). For which $p_i$ does the expected value of $t$ exist?

  • $\begingroup$ I think there will be a positive probability that $M^t(1)$ goes off to infinity. So the expected value will be undefined. $\endgroup$ Sep 1, 2016 at 23:14
  • $\begingroup$ @6005 Good point, modified the question $\endgroup$ Sep 1, 2016 at 23:16
  • $\begingroup$ Straightening out your terminology: what you are talking about is a Markov chain $X_t$ with state space $\mathbb{N}$ such that $X_{t+1}=X_t-1$ with probability $p_1$, $X_{t+1}=X_t$ with probability $p_2$, etc. Now you have $\tau=\inf \{ t : X_t=0 \}$ and you want to compute $E_1[\tau]$ where $E_1$ is expectation with respect to the chain started at the state $1$. $\endgroup$
    – Ian
    Sep 2, 2016 at 14:12
  • $\begingroup$ Sorry, slightly more precise: you have that configuration except when $X_t=0$ in which case $X_{t+1}=0$ as well. $\endgroup$
    – Ian
    Sep 2, 2016 at 14:19

1 Answer 1


For $p_1=1$, it trivially holds that $M(1)=0$. For $p_1<1$, no $t$ exists such that $M^t(1)=0$ holds with probability 1.

  • 1
    $\begingroup$ That second sentence is not actually a problem. The expected value could still exist as a finite number. $\endgroup$
    – Ian
    Sep 2, 2016 at 14:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .