0
$\begingroup$

The Normalized Mutual Information NMI calculation is described in deflation-PIC paper with the applicable formula copied to the screenshot shown below.

My question is specifically about the double summation of the numerator: why is it a double summation. My (apparently incorrect) intuition would be that the numerator would contain all of the elements that are common between the X and Y - as represented by the "L" and "H" summations in the formula. Then the common elements could be represented by a single summation over either one of the random variables (X or Y) and that filters for the corresponding member of the other cluster to have the same membership value (i.e. cluster). Instead the double summation is where I can use assistance for what that achieves.

enter image description here

enter image description here

$\endgroup$

1 Answer 1

1
$\begingroup$

The text is rather sloppy... First, it's $I(X;Y)$ , not $I(X,Y)$ (the notation is important). Second, "the mutual information between $X$ and itself is 1" made me quiver, until I understood that it actually meant "the normalized mutual information". Third, the factors inside the square root lack a minus sign - (yes, they cancel in the multiplication, but it's slightly confusing/ugly)

Regarding the numerator: the mutual information, in terms of the probabilities functions, is

$$I(X;Y)= \sum_{x,y}p_{X,Y}(x,y)\log \frac{p_{X,Y}(x,y)}{p_X(x) p_Y(y)} $$

Mapping $p_X(x) \to n_{\ell}/n $, $p_Y(y) \to n_{h}/n $ , $p_{X,Y}(x,y) \to n_{\ell,h}/n $ you get

$$I(X;Y) =\sum_{\ell,h} \frac{n_{\ell,h}}{n} \log \frac{n_{\ell,h} \, n}{n_{h} \, n_{\ell}} $$

So, ugly notations aside, it's right. The missing $1/n$ was cancelled with the same factor in the denominator of the complete NMI expression.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .