When evaluating a binary classifier, the basic data are as in this contingency table, where rows represent groundtruth value and columns represent the estimated value: $$ \begin{matrix} & + & - \\ \hline + & a & b \\ - & c & d \end{matrix} $$ where $a$ is the number of true positives, $c$ the number of false positives, etc. This information is often summarised on a ROC curve using the "true positive rate" and "false positive rate" statistics: $$ t = \frac{a}{a+b} \\ f = \frac{c}{c+d} $$ and from this we can calculate single values which summarise the performance, such as Area Under the Curve (AUC) (with only one contingency table, we can use $\text{AUC} = (t-f+1)/2$).
Another thing we could calculate is the mutual information between the estimated values and groundtruth values. This is a classic statistic that would tell us about the probabilistic relationship between our estimates and the groundtruth. In this case: $$ I = \frac{a}{n}\log{\frac{a n}{(a+b)(a+c)}} + \frac{b}{n}\log{\frac{b n}{(a+b)(b+d)}} + \frac{c}{n}\log{\frac{c n}{(c+d)(a+c)}} + \frac{d}{n}\log{\frac{d n}{(b+d)(c+d)}} $$ where $n=a+b+c+d$ and $0\log(0)=0$.
Calculating mutual information is straightforward if we know $a, b, c, d$. But what if we know only $t$ and $f$?
Clearly if all we know is $t$ and $f$ we can't directly calculate the mutual information, because the relative scale between $a$ and $b$ on one hand, and $c$ and $d$ on the other, isn't captured. But we must be able to lower-bound it. Here's a plot from random samples showing the relationship between AUC and mutual information:
I've tried a few ways of writing out the mutual information but not been able to get it into a form where $t$ and $f$ lead to interesting bounds. Can you?
