# How can I tell if dice are biased or unbiased, given a number of trials?

If I'm given the outcome of a number of dice rolls (say, 5 twos, 8 threes, etc), is there a way to assign a probability that the dice are biased or unbiased? If so, how?

Or alternatively, how can I say with a given level of confidence that the dice are biased or unbiased? I'm not even sure if these are the same question.

-
This is a standard statistics question, and I believe the standard answer is to use a Chi-squared test. I realize this isn't very explanatory, so I'll let someone else with more knowledge write a better answer. But if this kind of thing interests you, you might want to pick up a book on introductory statistics. I think there's a lot a background stats knowledge needed to fully answer this question. – Potato Dec 31 '12 at 1:17
@Potato: I took statistics in high school and college, but for some reason, that is the one math course that never sticks for me... I think the reason is that the derivations for chi-squared tests, or the equation for the bell-curve, or even for standard deviations are never explained... – BlueRaja - Danny Pflughoeft Dec 31 '12 at 2:38

Here is an account of what I understand to be the Bayesian approach, which I learned from Jaynes. Having read Jaynes, I am currently deeply suspicious of non-Bayesian approaches.

We'll restrict our attention to the slightly simpler case of a biased coin. Suppose the coin has some unknown probability $p$ of turning up heads, hence $1 - p$ of turning up tails. The first question is what your priors are regarding the distribution of possible values of $p$. For example, if you are already 100% confident that $p = \frac{1}{2}$, then no amount of evidence can shift this opinion. (This is why it is dangerous for Bayesians to be 100% confident of anything; see also 0 And 1 Are Not Probabilities.)

For simplicity, we'll use a uniform prior: that is, we'll assume initially that $p$ is equally likely to be any real number in $[0, 1]$. Now suppose we flip the coin $n$ times and observe $k$ heads and $n - k$ tails. Then by Bayes' theorem, the posterior distribution of $p$ has probability density function

$$\frac{ {n \choose k} x^k (1 - x)^{n-k} }{ \int_0^1 {n \choose k} x^k (1 - x)^{n-k} \, dx }.$$

This is a Dirichlet distribution. From here, to obtain a maximum likelihood estimate we need to find $x$ maximizing $x^k (1 - x)^{n-k}$. Taking logarithms and then derivatives, we find unsurprisingly that the maximum occurs when $x = \frac{k}{n}$. Note, however, that reporting only the maximum likelihood estimate is throwing away most of the information contained in the Dirichlet distribution, e.g. its variance. By doing more computations with the Dirichlet distribution you can write down, for example, the probability that $p$ is within a standard deviation of $\frac{k}{n}$.

In practice, the uniform prior is also unlikely to be an accurate description of your actual prior knowledge.

-
You're confusing a probability density with a probability. This special case of the Dirichlet distribution is the Beta distribution. If the density is $f(x)= \dfrac{x^k(1-x)^{n-k}}{\int_0^1 u^k(1-u)^{n-k}\,du}$, then $\Pr(p\in A)=\int_A f(x)\,dx$, but $\Pr(p=x)$ is just $0$. You can't have $\Pr(p=x)$ positive for uncountably many values of $x$. – Michael Hardy Dec 31 '12 at 5:57
@Michael: that was an accidental abuse of notation on my part. I'll edit it. – Qiaochu Yuan Dec 31 '12 at 6:17
I'm afraid this answer is over my head, I have no idea how to go from this to what I'm looking for, even after reading those links. – BlueRaja - Danny Pflughoeft Dec 31 '12 at 18:08
@BlueRaja: can you explain what you don't understand and also explain what you're looking for? The first issue is that from a Bayesian perspective you can't just ask for evidence of the coin being biased vs. unbiased because "biased" is not a sufficiently well-defined hypothesis for you to make predictions from. You need to specify priors, e.g. that you assign a probability of $\frac{1}{2}$ to $p = \frac{1}{2}$ and a probability of $\frac{1}{2}$ to $p$ being uniformly distributed in $[0, 1]$. – Qiaochu Yuan Dec 31 '12 at 20:55

This is a very classical problem and you will find many available sources online. Anything elementary on parameter estimation will include an analysis of a problem very much like what you describe (if not identical).

You do need to keep in mind that there are basically two common approaches (one more common than the other): Bayesian and non-Bayesian. Look introductory materia on "data analysis" for the latter or add "Bayesian" for the former.

A short summary of what you will find is that the two approaches quite often give very similar answers but not always. This is a subject of at times a heated debate in modern statistics.

-
"you will find many available sources online" - could you provide some links, then? Or at least reference some example text-books? – BlueRaja - Danny Pflughoeft Dec 31 '12 at 2:40
en.wikipedia.org/wiki/Bayesian_inference#Bayesian_inference is a pretty good account of the Bayesian approach. en.wikipedia.org/wiki/Sequential_analysis quickly reviews the classical case. – Ittay Weiss Dec 31 '12 at 2:51