In season 14 of the Top Chess Engine Championship, Stockfish defeated Leela 50.5-49.5 in a 100-game match (the closest possible winning margin). Using an elo calculator such as this one, we can see that the elo difference between the two engines was +3.

However it must be that this isn't a statistically significant result - after all the match was only 100 games, and it's the smallest possible winning margin. Therefore the error bars on this calculation should be at least 3. The calculator doesn't give the error bars though, and I can't find any equations on how to calculate the error. How do you calculate the error on this estimate?

I originally asked this on Chess.SE but was redirected here.

If it matters, in chess, if you win a game you score 1 point, if you draw you get 0.5 points, and if you lose you get zero. Stockfish's win was +10 =81 -9, i.e. it won 10 games, lost 9, and drew 81.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.