conditional probability with vs. without replacement independence?

Question

I have $2$ jars, one of which is chosen randomly ($.5$ chance)
jar 1: $8$ blue marbles (b), $4$ red marbles (r)
jar 2: $15$ blue marbles, $3$ red marbles

In a situation WITH replacement I know the probability of the first pick being a blue marble given that the second pick (from the same jar as the first pick) is a red marble =
$$ P(p_1 = b \;|\; p_2 = r) = P(p_1=b \;|\; jar=1)P(jar=1 \;|\; p_2=r) + P(p_1=b \;|\; jar=2)P(jar=2 \;|\; p_2=r) $$

But what does the same probability WITHOUT replacement look like? I know this might be an easy question but I have tried a lot and nothing seems to add up. Thank in advance!

Answer 1

The complete formula (for both problems, with or without replacement) is

\begin{multline} P(p_1 = b \mid p_2 = r) = P(p_1=b \mid jar=1,\ p_2 = r)P(jar=1 \mid p_2=r) + {} \\ P(p_1=b \mid jar=2,\ p_2 = r)P(jar=2 \mid p_2=r) \end{multline}

When you assume replacement, you can get away with the formula you wrote because the two draws are independent of each other (once the jar has been selected): $$ P(p_1=b \mid jar=1,\ p_2 = r) = P(p_1=b \mid jar=1,\ p_2 = b) = P(p_1=b \mid jar=1). $$

If you have trouble conceiving how to compute $P(p_1=b \mid jar=1,\ p_2 = r),$ remember that for any combination of balls that might be drawn (such as "one red, one blue"), the balls are equally likely to come out in any order. Also, the prior probability for each ball (not conditioned on the balls before or after) is the same as for any other ball. From these facts it follows that $$ P(p_1=b \mid jar=1,\ p_2 = r) = P(p_2=b \mid jar=1,\ p_1 = r), $$ even though people often find the probability on the left much more difficult to think about than the probability on the right.