# Probability from a collection of independent predictions

Reframed question:

Team A and Team B will be playing each other in a sporting event where there is one winner and one loser; a tie is impossible. A man, wanting to make money from placing a bet on the winner, consults with three omniscient seers. These seers know the outcome of the game already, being omniscient.

The first seer informs the man (truthfully) that he randomly lies 20% of the time when telling the outcome of a sporting event, telling the truth the rest of the time.

The second seer says (truthfully) that he is just like the first, except he lies 40% of the time.

The third seer says (truthfully) that he is just like the first and second seers, except he lies 70% of the time.

The first seer tells the man that team A will win. The second seer tells the man that team A will lose. The third seer tells the man that team A will win.

What is the probability that team A will win?

Each seers' determination of whether to lie is independent of the others. You can imagine that each seer rolls a ten sided die to make their decision of whether to lie.

After the above question is answered, I am curious if someone can come up with a general formula for any number of seers with any probability of lying, with any set of predictions.

-
The term "independent" might be misleading, since the seers are linked by the truth of the outcome. – Muhd Mar 10 '11 at 8:06
@Muhd: Somewhat ironically, I'd written a long answer dealing with assumptions about independence in your original question before you reframed it, but in the reframed version, independence is unproblematic, since it only refers to the random determinations whether to lie, which can perfectly well be independent even though the results of lying or not lying will of course be correlated by the actual outcome. – joriki Mar 10 '11 at 8:12
@joriki Sorry about that. I'm sure your thoughts on independence would have been lovely. I realized that in real world scenarios people often base their predictions on similar or related data, so an assumption of complete independence would be very unrealistic and would lead to a misleading result. This makes me wonder to what extent the formula below is useful. It might do well in say, a stock market setting, where different traders might make predictions on different aspects of a company. But even then there are all sorts of correlations between the various company stats. – Muhd Mar 10 '11 at 9:54
@Muhd: That, in essence, was my answer to your original question :-). – joriki Mar 10 '11 at 9:59
I suppose in the stock example it could be useful, even in the case of dependence between the predictors, to compare predictions from the same set of traders for different companies to determine which is better. The actual number you get would of course not be very accurate, and would tend toward the extremes (toward 0 or 1), but for relative comparisons it would be fine I think. – Muhd Mar 10 '11 at 10:11

There's not enough information to give a probability for team A winning. You can see this by imagining that you know a priori that the probability for team A to win is $0$ or $1$ -- then this won't change if you listen to the seers. The information from the seers can only modify your a priori knowledge, insofar as it's incomplete, but it can't give you a probability for team A to win if you don't already have some assumption about that probability.

Having said that, on the assumption that a priori the teams are equally likely to win (a "flat prior"), it's straightforward to calculate the conditional probability of team A winning given the information from the seers:

For team A to win, the seers would have had to roll dice with probabilities 80%, 40% and 30%, respectively. For team A to lose, they would have had to roll dice with probabilities 20%, 60% and 70%, respectively. So the conditional probability for team A to win given the information from the seers is

$$\frac{0.8\cdot0.4\cdot0.3}{0.8\cdot0.4\cdot0.3 + 0.2\cdot0.6\cdot0.7}\approx 53\%\;.$$

Update in response to the added last sentence in the question:

The generalization to any number of seers with any probability of lying with any set of predictions is straightforward: Given $n$ seers with probability $p_i$ of lying and predictions $b_i\in\{0,1\}$ of team A winning, again assuming a 50/50 prior, the probability of team A winning is

$$\frac{\prod_i \left(b_i(1-p_i)+(1-b_i)p_i\right)}{\prod_i \left(b_i(1-p_i)+(1-b_i)p_i\right)+\prod_i \left(b_ip_i+(1-b_i)(1-p_i)\right)}\;.$$

Further update:

You can also generalize this to an arbitrary prior by taking into account the prior probabilities. Assuming that a priori the probability for team A to win is $p$, the conditional probability given the information from the seers is

$$\frac{p\prod_i \left(b_i(1-p_i)+(1-b_i)p_i\right)}{p\prod_i \left(b_i(1-p_i)+(1-b_i)p_i\right)+(1-p)\prod_i \left(b_ip_i+(1-b_i)(1-p_i)\right)}\;,$$

and you recover the above result by setting $p=1/2$. You can also get the two cases where your a priori knowledge is complete and hence isn't modified by the new information from the seers by choosing $p=0$ or $p=1$, in which case the conditional probability also comes out as $0$ and $1$, respectively.

-
Very nice answer. The assumption of a flat prior is exactly what I had in mind. – Muhd Mar 10 '11 at 10:14