Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top


Suppose that the national mint mints coins with bias $p_i \sim \rm{Beta}(A,B)$ for some unknown constants $A$, $B$.

Given $n$ coins, you flip each coin a certain number of times. Coin $i$ comes up heads $a_i$ times and tails $b_i$ times. (The flips are clearly exchangeable, so we don't need the actual sequence of flips.)

If I did the math right, I figured out that the likelihood of $A, B$ given the statistics $(a_i, b_i)$ is:

$$L(A,B \mid a_i, b_i) = \frac{\prod_i\rm{Beta}(A + a_i, B + b_i)}{\rm{Beta}(A, B)^n}.$$

(Edit in 2012: This is the product of $n$ Pólya–Eggenberger urn schemes.)

I would like to summarize the statistics $(a_i, b_i)$ even further, into a finite set of numbers if possible.


Can $\prod_i\Gamma(A + a_i)$ be written using sufficient statistics calculated from the $a_i$?


For example, $\prod_i\exp(A + a_i)$ can be written $\exp(nA + S)$ where $n, S$ are sufficient statistics calculated from the $a_i$.

share|cite|improve this question
If this question interests you, please help me tag it. – Neil G Jun 28 '11 at 4:37
up vote 2 down vote accepted

I think I would agree with your expression for the likelihood, although this approach is not without problems (see below).

Notice, that since $a_i$ and $b_i$ are integers, the likelihood is just a rational function in $A$ and $B$: $$ L(A,B \mid a,b) = \prod_{i=1}^n \left( \frac{ \prod\limits_{k=0}^{a_i-1} (A+k) \prod\limits_{m=0}^{b_i-1} (B+m) }{\prod\limits_{j=0}^{a_i+b_i-1} (A+B+j)}\right) $$ The maximum likelihood equations are easily seen to be $$ \begin{eqnarray} \sum_{i=1}^n \sum_{k=0}^{a_i-1} \frac{1}{A+k} &=& \sum_{i=1}^n \sum_{j=0}^{a_i+b_i-1} \frac{1}{A +B+j} \\ \sum_{i=1}^n \sum_{m=0}^{b_i-1} \frac{1}{B+m} &=& \sum_{i=1}^n \sum_{j=0}^{a_i+b_i-1} \frac{1}{A +B+j} \end{eqnarray} $$ Given that $A>0$ and $B>0$ these equations become polynomial equations in $A$ and $B$ with, hopefully, a unique positive root. However, checks in Mathematica for small values of $n$ and $a_i$, $b_i$ indicate that the resulting system admits no positive finite solutions. enter image description here

It is easily seen, that infinite $A$ and $B$ solve MLE equations. Let's fix $r=A/B$. Then, in the limit of infinite $A$ and $B$, equations reduce to $$ \begin{eqnarray} \sum_{i=1}^n a_i &=& \sum_{i=1}^n \frac{r}{r+1} (a_i+b_i) \\ \sum_{i=1}^n r b_i &=& \sum_{i=1}^n \frac{r}{r +1} \cdot (a_i+b_i) \end{eqnarray} $$ which gives $$ r = \frac{\sum_{i=1}^n a_i}{\sum_{i=1}^n b_i} $$

This result is confirmed numerically by the following heuristic. Use $a_i$ and $b_i$ to estimate head probability $p_i$ for each coin using ML estimator $p_i = \frac{a_i}{a_i+b_i}$. Then use ML estimation for beta distribution to determine $\alpha$ and $\beta$ from the data $\{p_1, \ldots, p_n \}$.

enter image description here

I do not have a good explanation as to why ML estimation can not seem to determine both $\alpha$ and $\beta$. This may be worth a separate question.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.