Probability that the red ball is the last remaining ball I have the following problem, I've been thinking about it for a while but I can't get any further with it:
You have 80 red balls and 20 blue balls in a box. You pick a random ball

*

*if the ball is red, you throw it out

*if the ball is blue, you put it back into the box, and pick another random ball and throw that one out regardless of colour.

I've seen people just write code and they came up with the probability of around 0.34 if I remember it correctly, but I'm sure there's an actual mathematical way of solving this in terms of combinatorics.
How would I go about solving this? My first guess was calculating the probability of picking out the blue balls (20/100, then 19/99) but that's faulty because it assumes you're consistently picking out blue balls and then subsequently throwing them out.
Thanks in advance for any help.
 A: We can describe the state of the system after any move as an ordered pair $(r,b)$, meaning that $r$ red balls and $b$ blue balls remain.  Let $P_{r,b}$ be the probability that the system visits state $(r,b)$ at some time.  We are asked for $P_{1,0}$.  We are told that the system starts in state $(80,20)$.
The system can be described as an absorbing Markov chain, but because the system never visits a state twice, we don't need all the machinery described on the wiki page.
How can we get to state $(1,0)$?  The previous state must have been $(2,0)$ or $(1,1)$.  From $(2,0)$ we can only go to $(1,0)$.  From $(1,1)$ we go to $(1,0)$ only if we pick the blue ball twice.  That is, $$P(1,0)=P(2,0)+\frac14P(1,1)$$
This works in general.  To get to state $(r,b)$ we must come from $(r+1,b)$ or from state $(r, b+1)$.  To come from $(r+1,b)$, we must either choose the red ball, or first choose the blue ball and then the red ball.  To come from $(r, b+1)$, we must choose the blue ball twice.  Of course, there are adjustments that need to be made when $r=80$ or $b=20$.
Of course, you'd want to write a computer program to calculate the probabilities, but you can calculate the precise answer.  You can even write it in a fairly compact form, using the formula for Markov chains, although this isn't very useful.
The program starts with $P(80,20)=1$.  It then computes $P(79,20)$ and $P(80, 19)$ as described above.  Then it computes all $P(r,b)$ for all cases where $r+b=98$, then the cases where $r+b=97$ and so on, down to $r+b=1$.
EDIT
Are you sure the probability was about $.34$?  That's an order of magnitude more than I get.  I wrote a little python script:
from collections import defaultdict
P = defaultdict(float)
P[80, 20] = 1
for balls in range(99,0,-1):
    for b in range(balls-80, 1+min(20,balls)):
        r= balls - b
        P[r, b] += (r+1)/(balls+1)*P[r+1,b]
        P[r, b] += (b*(r+1))/(balls+1)**2*P[r+1,b]
        P[r, b]  += ((b+1)/(balls+1))**2*P[r,b+1]
print('red', P[1,0])
print('blue', P[0,1])  

This produced the output
red 0.03809523809523809
blue 0.9619047619047617

I haven't been able to find a mistake in the script.  I may write a simulation to cross-check.
EDIT
Simulation of $100,000$ trials gave $0.03952$, so I'm confident that $.038$ is correct.
