probability calculation for failure compensation

Question

I try to calculate the probablity of the following events:

At each trial, the probability of success is given by $s_1$. If the trial is failed 9 times in a row, the probability of success is changed by $s_2$. When the event is succeed at trial $n$, the probability at $n+1$ is changed to $s_1$, and remain as $s_1$ untill it fails 9 times.

I wish to know the compensated probability at $n$ trials.

I calculate the probability at $n$ such as:

$$ p_n=\prod_{j=n-8}^{n-1} (1-p_{j}) \times s_2 + s_1 ,\ \ \text{for} \ n>9 $$

$$ p_n=s_1 \ \ ,\text{else} $$

But it seems it's not true. I coded a program to calculate this. When the $s_1$=0.94 % , and $s_2$=10 %, the probability is converged to around 1.5 %.

How can i set an equation to solve this?

Answer 1

This is a Markov chain with 10 states. States 0-8 reflect the number of consecutive failures at this point. In each of these the probability of success is $s_1,$ and on failure, you transition from state $n$ to state $n+1$. On success, you transition to state 0. State 9 is the sate where the probability of success is $s_2.$ In state 9, on failure you stay in state 9, and on success you transition to state 0.

The initial state is state $0$, with probability $1.$

If $P$ is the transition matrix, you want to look at $XP^n,$ where X is the row vector $(1,0,...,0)$ This gives you the probability of being in each state after $n$ trials, from which you can easily determine the probability of success at the next trial. If $p$ is the probability of being in state 9, then the probability of success on the next trial is $ps_2+(1-p)s_1.$

EDIT Re-reading your question, it seems as though you may actually be interested in $\lim_{n\to\infty}p_n$. In that case, you want to lok at the steady-state probability vector $\pi$ which is the solution to $$\pi=\pi P$$ subject to the condition that the sum of the elements of $\pi$ is $1$. In that case, you don't have to worry about the initial state, since the chain will converge to the steady state no matter where it starts. You calculate the probability of success as above. If I get some time later today, I'll compute it for $s_1 = .094, s_2=.1,$ so we can compare answers.

ADDENDUM Actually, this turn out to be easy to solve symbolically, at least if you can avoid the idiot mistakes I kept making. Let the sattes be as above, and let $p_k$ be the stationary probability that the chain is in state $k, 0\le k \le 9.$ For $1\le k\le 8,$ the chain is in state $k$ precisely when the previous trial was a failure in state $k-1,$ so $$ p_k=(1-s_1)^k\implies p_k=(1-s_1)^kp_0, 0\le k\le 8$$ The chain is in state $9$ precisely when the previous trial was a failure in state $8$ or $9$ so that$$ p_9=(1-s_1)p_8+(1-s_2)p_9=(1-s_1)^9p_0+(1-s_2)p_9\implies\\ p_0=\frac{s_2p_9}{a}, \text{ where } a=(1-s_1)^9 $$ We must have $\sum{p_k}=1,$ so $$ 1=\sum_{k=0}^9{p_k}=\sum_{k=0}^8{(1-s_1)^kp_0}+p_9=\frac{1-a}{s_1}\frac{s_2p_9}{a}+p_9\implies\\ p_9=\frac{as_1}{(1-a)s_2+as_1},\text{ where } a = (1-s_1)^9 $$ Now the probability of success is $$ s_2p_9+s_1(1-p_9)=\boxed{\frac{s_1s_2}{(1-a)s_2+as_1}}$$ with $a$ as above.