# The limiting case of a discrete probability problem

Say there are three jars, $j_1, j_2, j_3$ filled with different binary sequences of length two.

The distribution of the binary sequences in each of the jars is given by the $p_i^k(1-p_i)^{n-k}$, where $p_i = \frac{i}{m + 1}$ where $m$ is the number of jars, $i$ is the jar index, $k$is number of 1's and $n$ is the length of the string.

So for three jars we have $p_1 = 0.25, p_2 = 0.5$, and $p_3 = 0.75$ for $j_1, j_2, j_3$ respectively.

Here are the sequences and their probabilities for $j_1$ with $p_1 = 0.25$:

\begin{align*} P(00) = 9 / 16 \\ P(10) = 3 / 16 \\ P(01) = 3 / 16 \\ P(11) = 1 / 16. \end{align*}

If I tell you that I have selected a binary sequence and the first element is $1$ what is the E($p_i$)?

Well, this can be calculated by looking at each of the jars and adding up the probability of candidate sequences times the value of $p_i$.

Edit: I wasn't normalizing this conditionally space properly. I'm skipping a step which I'll explain, someone wants.

\begin{equation*} E(p_i) = (4/24 * 1/4) + (8/24 * 1/2) + (12/24 * 3/4) = 14 / 24 = 0.58. \end{equation*}

So the question is ... what is $E(p_i)$ when the numbers of jars goes to infinity (or alternatively, when $p$ can take on values between $0$ and $1$)? Also what happens when the size of the binary strings goes to infinity? Does it have an effect on the outcome? If it does, does the order we take the limits change the answer?

And most importantly what is the general case for when I have $s$ 1's and $r$ $0$'s?, with a continuous $p$ from $0$ to $1$ and infinite sequences?

-
If I tell you that I have selected a binary sequence and the first element is 1 what is the expected value of X... What is X? I don't see where you've defined it. – Isaac Jul 27 '10 at 19:19
E(value of $p_i$). It might be confusing where the probabilities in the expected value computation came from which I can explain. The other point is that when we are calculating the expected $p_i$ it is just a value associated with the jars, it is not the probability that you picked a binary sequence from the jar. – Jonathan Fischoff Jul 27 '10 at 19:33
That was confusing, my bad – Jonathan Fischoff Jul 27 '10 at 19:42
In retrospect, after finally catching my error in the calculation of P(1...), since we only care about the probability that the first digit is 1 and the rest of the digits (however many there may be) are free, P(1...) should be the probability of a single digit being 1, which is p_i, regardless of n. – Isaac Jul 27 '10 at 22:11
Yes, that's what I would expect, since this question was attempt at rephrasing this one: math.stackexchange.com/questions/695/…. I'm not positive they are equivalent, but I think they are... – Jonathan Fischoff Jul 27 '10 at 22:27

## 1 Answer

First, sticking to strings of length n=2:

$P(10)=p_i (1-p_i)=\frac{i}{m+1}\left (1-\frac{i}{m+1} \right )=\frac{i(m+1-i)}{(m+1)^2}$ and $P(11)=p_i^2=\left (\frac{i}{m+1} \right )^2=\frac{i^2}{(m+1)^2}$;

$E(p_i)=\sum_{i=1}^{m}p_iP(p_i)=\sum_{i=1}^{m}\frac{i}{m+1}\left(\frac{P(10)+P(11)}{\sum_{i=1}^{m}(P(10)+P(11))} \right )$ $=\frac{\sum_{i=1}^{m}\frac{i}{m+1}\left(\frac{i(m+1-i)}{(m+1)^2}+\frac{i^2}{(m+1)^2} \right )}{\sum_{i=1}^{m}\left(\frac{i(m+1-i)}{(m+1)^2}+\frac{i^2}{(m+1)^2} \right )}$ $=\frac{\sum_{i=1}^{m}\frac{i}{m+1}\left(i(m+1-i)+i^2\right )}{\sum_{i=1}^{m}\left(i(m+1-i)+i^2\right )}$ $=\frac{\sum_{i=1}^{m}i^2}{(m+1)\sum_{i=1}^{m}i}$ $=\frac{\left(\frac{m(m+1)(2m+1)}{6} \right )}{(m+1)\frac{m(m+1)}{2}}$ $=\frac{2m+1}{3(m+1)}$;

$\lim_{m\to\infty}E(p_i)=\lim_{m\to\infty}\frac{2m+1}{3(m+1)}=\frac{2}{3}$

For general n:

$P(1...)=p_i\sum_{k=0}^{n-1}\left({n-1 \choose k}p_i^k(1-p_i)^{n-1-k} \right )=p_i$ (yes, P(1...) does not depend on n at all!);

$E(p_i)=\sum_{i=1}^{m}p_iP(p_i)=\sum_{i=1}^{m}p_i\left(\frac{P(1...)}{\sum_{i=1}^{m}P(1...)} \right )=\sum_{i=1}^{m}p_i\left(\frac{p_i}{\sum_{i=1}^{m}p_i} \right )$ $=\frac{\sum_{i=1}^{m}p_i^2}{\sum_{i=1}^{m}p_i}=\frac{\sum_{i=1}^{m}\left(\frac{i}{m+1} \right )^2}{\sum_{i=1}^{m}\frac{i}{m+1}}$ $=\frac{\sum_{i=1}^{m}i^2}{(m+1)\sum_{i=1}^{m}i}=\frac{2m+1}{3(m+1)}$ (no surprise there, given that $P(1...)$ doesn't depend on n);

$\lim_{m\to\infty}E(p_i)=\lim_{m\to\infty}\frac{2m+1}{3(m+1)}=\frac{2}{3}$

-