# Find expected value of length of trajectory under probabilistic mapping

I'm a bit of a newbie when it comes to probability theory/terminology, so please excuse any errors on my part. Note that I ask this out of curiosity, so a full answer would be appreciated. Here goes the problem:

Define the probabilistic map $M:\Bbb N\mapsto\Bbb N$ as follows:

$M(n)=\begin{cases}n-1&\text{probability}~p_1\\n&\text{probability}~p_2\\n+1&\text{probability}~p_3\\n+2&\text{probability}~p_4\\n+3&\text{probability}~p_5\\n+4&\text{probability}~p_6\end{cases}$

with $p_1$, ..., $p_6$ constant and $p_1+\cdots+p_6=1$. Now, define $t$ as the first value such that $M^t(1)=0$ (with $M^t$ denoting repetition). For which $p_i$ does the expected value of $t$ exist?

• I think there will be a positive probability that $M^t(1)$ goes off to infinity. So the expected value will be undefined. – 6005 Sep 1 '16 at 23:14
• @6005 Good point, modified the question – LegionMammal978 Sep 1 '16 at 23:16
• Straightening out your terminology: what you are talking about is a Markov chain $X_t$ with state space $\mathbb{N}$ such that $X_{t+1}=X_t-1$ with probability $p_1$, $X_{t+1}=X_t$ with probability $p_2$, etc. Now you have $\tau=\inf \{ t : X_t=0 \}$ and you want to compute $E_1[\tau]$ where $E_1$ is expectation with respect to the chain started at the state $1$. – Ian Sep 2 '16 at 14:12
• Sorry, slightly more precise: you have that configuration except when $X_t=0$ in which case $X_{t+1}=0$ as well. – Ian Sep 2 '16 at 14:19

For $p_1=1$, it trivially holds that $M(1)=0$. For $p_1<1$, no $t$ exists such that $M^t(1)=0$ holds with probability 1.