Using head-to-head results and Bayes' Theorem to modify predictions of sport/game contests that are initially derived from Elo-type ratings

Question

I am working on an extension of the Glicko2 rating system to use in predicting the outcome of sport/game contests that uses the actual head-to-head results of previous meetings of competitors to provide more information in calculating the probability of each side winning a particular game. What I am trying to do is say that the ratings provided by Glicko2 give the prior probabilities of each side winning, and use the head-to-head results as the Bayesian updating data. That is, I am looking for the probability of a side winning given the head-to-head results, using the probability from the rating system as my prior. In order to use Bayes' Theorem, however, I need to know things that I'm not really sure how to figure out even with access to all the data I have. Specifically, I need to know the probability that the head-to-head data would be observed "under the hypothesis" that a certain side wins the next match. How exactly to do this is not particularly simple. I was initially unsure of even anything about how to calculate these quantities and asked a question at a general-information intellectual message board, and no one there could help me but suggested I try here. My thread there is the top google result (at least for me) for "updating rating systems probabilities using head-to-head results", and all other results are not even close to envisioning the situation that I have constructed for myself.

I'm not even sure what the "probability of observing the data given the hypothesis" even means in this situation. In other Bayes' Theorem problems I've looked at, those probabilities are given or very simple to find. I thought I had some sort of answer, but after coding it up noticed that it was returning absurd results, specifically that a side with a 75% win chance as calculated from ratings and a 60% win rate in prior contests against the opposing side would win less than 60% of the time. Clearly there is something wrong with my calculations there. I was modeling the probabilities of the data given each side winning as though the data were being drawn from a binomial distribution with probability as implied by what the two sides' ratings would be updated to given that side winning. I also tried taking the binomial distribution's probabilities from ratings at the time of the previous contests, but the absurd results were still there, and it was far less obvious how to modify the binomial probability based on each side winning.

Where is the mistake in my approach here? How should I be finding the conditional probabilities that are reversed from the ones that I'm actually trying to calculate? Is this even not actually a Bayes' Theorem application because such conditional probabilities are not obvious, and I'm better off looking for the probability I'm most interested in a more direct manner?

Answer 1

Let's say your rating system says A consistently has a $60\%$ chance of winning against B. You have the following match data available: in four matches, the victors were A, A, B, A. In reality, the results of matches are not independent of each other (contestant confidence and other factors are in effect) but let's assume they are.

The prior odds of A winning were $3:2$. We multiply this by the odds of the four matches actually having those results given the prior odds, i.e. $3:2 \times 3:2 \times 3:2 \times 2:3 \times 3:2 = 162: 48 = 27:8$. This gives a normalized probability of A winning as $27/35 \approx 77\%$.

This is by no means a very good model of the probabilities, because:

1. As mentioned, matches are not independent of each other
2. The prior probability given by the rating system is perhaps not completely independent of the match results (can you provide more details about the rating system in your question?)
3. Many factors affect match outcomes, including (in real life) weather, location, winning/losing streaks, support from fans/crowd, etc. which we are ignoring

However, I think this analysis gives a decent estimate given the information you have provided. To get a more intuitive grasp of Bayesian reasoning, you should check out this website.