I'm a bit of a newbie when it comes to probability theory/terminology, so please excuse any errors on my part. Note that I ask this out of curiosity, so a full answer would be appreciated. Here goes the problem:

Define the probabilistic map $M:\Bbb N\mapsto\Bbb N$ as follows:


with $p_1$, ..., $p_6$ constant and $p_1+\cdots+p_6=1$. Now, define $t$ as the first value such that $M^t(1)=0$ (with $M^t$ denoting repetition). For which $p_i$ does the expected value of $t$ exist?

  • $\begingroup$ I think there will be a positive probability that $M^t(1)$ goes off to infinity. So the expected value will be undefined. $\endgroup$ – 6005 Sep 1 '16 at 23:14
  • $\begingroup$ @6005 Good point, modified the question $\endgroup$ – LegionMammal978 Sep 1 '16 at 23:16
  • $\begingroup$ Straightening out your terminology: what you are talking about is a Markov chain $X_t$ with state space $\mathbb{N}$ such that $X_{t+1}=X_t-1$ with probability $p_1$, $X_{t+1}=X_t$ with probability $p_2$, etc. Now you have $\tau=\inf \{ t : X_t=0 \}$ and you want to compute $E_1[\tau]$ where $E_1$ is expectation with respect to the chain started at the state $1$. $\endgroup$ – Ian Sep 2 '16 at 14:12
  • $\begingroup$ Sorry, slightly more precise: you have that configuration except when $X_t=0$ in which case $X_{t+1}=0$ as well. $\endgroup$ – Ian Sep 2 '16 at 14:19

For $p_1=1$, it trivially holds that $M(1)=0$. For $p_1<1$, no $t$ exists such that $M^t(1)=0$ holds with probability 1.

  • 1
    $\begingroup$ That second sentence is not actually a problem. The expected value could still exist as a finite number. $\endgroup$ – Ian Sep 2 '16 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.