Binomial probability mass function with selective variable reflipping What is the probability mass function $P(N, k, p)$ where


*

*$N = \{N_1, N_2, \ldots, N_m\}$ is a sequence representing how many coins can be flipped (and potentially reflipped if $m > 1$)

*$k$ is number of successes

*$p$ is the probability of success per coin flipped

*There are a total number of $N_1$ coins, thus $N_1 \geq \text{max}(N_2, N_3, \ldots, N_m)$

*Only $\text{min}(N_i, N_1 - k_{i-1})$ coins are reflipped on each reflip, where $k_{i-1}$ is the number of successes before the reflip

*$P(N, 0 \leq k \leq N_1, p) = \sum_{k=0}^{N_1}P(N, k, p) = 1$


Example: Consider $N = \{3, 1, 2\}$


*

*$N_1 = 3$ coins are initially flipped

*If 3 successes occur, no coins are reflipped

*If 2 successes occur on the first flip, $k_1 = 2, \text{min}(N_2 = 1, N_1 - k_1 = 1) = 1$ coins are reflipped


*

*If 0 success occurs, $\text{min}(N_3 = 2, N_1 - k_2 = 1) = 1$ coin is reflipped

*if a success occurs, $\text{min}(N_3 = 2, N_1 - k_2 = 0) = 0$ coins are reflipped


*If 1 success occurs on the first flip, $\text{min}(N_2, N_1 - k_1) = 1$ coin is reflipped


*

*If 0 successes occur on the reflip, $\text{min}(N_3, N_1 - k_2) = 2$ coins are reflipped

*If 1 success occurs, $\text{min}(N_3, N_1 - k_2) = 1$ coin is reflipped


*If 0 successes occur on the first flip, $\text{min}(N_2, N_1 - k_1) = 1$ coin is reflipped


*

*If 0 or 1 success occurs on the reflip, $\text{min}(N_3, N_1 - k_2) = 2$ coins are reflipped



I've tried the following:
$$
\text{Let}\ P_b(n, k, p) = \binom{n}{k}p^k(1-p)^{n-k}\\
P(N, k, p) =
P_b(N_1, k, p) + \sum_{k_1=0}^{k-1}P_b(N_1, k_1, p)\left(
 P_b(N_2, k-k_1, p) + \sum_{k_2=0}^{k-k_1-1} \ldots
\right)
$$
However I know this is wrong because $P(\{3, 3\}, 1 \leq k \leq 3, 0.5) > 1$

Original questions:
What is the probability mass function $P(n, k, p, R)$ where


*

*$n$ is the number of coins flipped

*$k$ is the target number of successes

*$p$ is the probability of coin flip success

*$R = \{R_1, R_2, ..., R_m\}$ is a sequence denoting how many coins can be reflipped and may be empty. If the first flip doesn't succeed $R_1$ coins cans be reflipped and so on.


Example: $P(3, 2, 0.5, \{1, 2\})$ is the probability $k = 2$ success occur when $n = 3$ coins are flipped (and reflipped).


*

*If 3 successes occur on the first flip, this is considered a failure and no coins are reflipped

*If 2 successes occur on the first flip, no coins are reflipped

*If 1 success occurs on the first flip, 1 coin is reflipped (according to the sequence)


*

*If 0 successes occur on the reflip, 2 coins are reflipped to try to get the second success (according to the sequence and since we currently have 1 success and 2 failures)

*Otherwise we now have 2 successes and no coins are further flipped


*If 0 successes occur on the first flip, 1 coin is reflipped


*

*If 0 successes occur on the reflip, 2 coins are reflipped to try to get 2 successes

*If 1 success occurs on the reflip, 2 coins are reflipped to try to get one more success


 A: This is an answer to the original question, edited by the OP to reflect some of the notational changes in his rewrite of the question. Note, however, that the summation indices $k_i$ used here count the number of successes in the last flip and thus don't coincide with the variables introduced in the rewrite of the question, which represent the total number of successes up to the flip.

The general formula would be
$$
\binom {N_1}kp^k(1-p)^{N_1-k}+\sum_{k_1=0}^{k-1}\binom {N_1}{k_1}p^{k_1}(1-p)^{N_1-k_1}\left(\binom{\min(N_2,N_1-k_1)}{k-k_1}p^{k-k_1}(1-p)^{\min(N_2,N_1-k_1)-k+k_1}+\sum_{k_2=0}^{k-k_1-1}\cdots\right)\\
=p^k\left(\binom {N_1}k(1-p)^{N_1-k}+\sum_{k_1=0}^{k-1}\binom {N_1}{k_1}(1-p)^{N_1-k_1}\left(\binom{\min(N_2,N_1-k_1)}{k-k_1}(1-p)^{\min(N_2,N_1-k_1)-k+k_1}+\sum_{k_2=0}^{k-k_1-1}\cdots\right)\right)\;,
$$
where we can factor out $p^k$ since for an overall success the total number of successes is always $k$. In your example, this would yield
$$
p^2\left(\binom32(1-p)+\sum_{k_1=0}^1\binom3{k_1}(1-p)^{3-k_1}\\\left(\binom1{2-k_1}(1-p)^{k_1-1}+\sum_{k_2=0}^{1-k_1}\binom1{k_2}(1-p)^{1-k_2}\binom2{2-k_1-k_2}(1-p)^{k_1+k_2}\right)\right)\\
=p^2\left(3(1-p)+(1-p)^3\left((1-p)+2(1-p)\right)+3(1-p)^2\left(1+(1-p)2(1-p)\right)\right)\\
=p^2\left(3(1-p)+3(1-p)^2+9(1-p)^4\right)\;.
$$
Arguably this could be obtained more easily by just looking at the individual case than from the broad generalisation.
