# confused about joint mutual information

I have a difficulty understanding 'joint mutual information'

The expressions like $I(X,Y;B)$ are not understood.

Is there an good example to understand joint mutual information?

Actually, I want to know the following text. (Michel 2004, The Extraction and Use of Informative Features for Scale Invariant Recognition)

In this context, the expression like $I(A,B;C)$ appears. I want to understand this. But there is a little material to study.

In it, $I(X;Y)$, $I(X;Y|Z)$ and $I(X_1,X_2;Y)$ appeared. It is hard to understand the differences.

• Can you give us some background? Have you seen a definition of "joint mutual information" somewhere? If so, what was it? – dfeuer Sep 2 '13 at 8:20
• OK. I've updated. – jakeoung Sep 2 '13 at 9:30

Mutual information relates two random variables $X$ and $Y$. The variables are usually separated by a semicolon, and the relation is symmetric. So when you read $I(X;Y)$ you should think as $\{X\} \overset{I}{\longleftrightarrow}\{Y\}$
(BTW, the main relations are $I(X;Y)=H(X)-H(X|Y)=H(Y)-H(Y|X)=I(Y;X)$, but you probably already knew this).
When we write $I(X_1,X_2;Y)$ we (usually) mean that $X_1$ and $X_2$ should be regarded as a composite (multivariate) variable, and we are computing the mutual information of this composite variable with $Y$. That is, $\{X_1,X_2\} \overset{I}{\longleftrightarrow}\{Y\}$
Regarding $I(X;Y |Z)=H(X|Z)-H(X |Y,Z)$, this is again a mutual information between $X$ and $Y$, only that conditioned to knowledge of $Z$. It's, again symmetric, so $I(X;Y |Z)=I(Y;X |Z)$.
So, if you were confused about the "precedence" in the above expressions: let's say that "$,$" (composition of variables) binds stronger than "$;$" (mutual information), so that $I(X_1,X_2;Y)$ should be read as $I( (X_1,X_2);Y)$; and the later binds stronger than "$|$" (global conditioning).