Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am confused about the difference which the use of ";" and "," causes in the following expressions for defining mutual information between X and Y

$ I(X ; Y) $ and $ I(X , Y) $

share|cite|improve this question
There is no difference. The mutual information is defined between two random variables. – Euclean Nov 12 '12 at 8:26

I believe it is because $I(X;Y) = I(Y;X)$. Mutual information does not measure a directional information flow, but how much $X$ can tell you about $Y$, and vice versa.

In addition, sometimes it is necessary to represent MI between $(X,Y)$ and $Z$, in which case $I(X,Y;Z)$ would be a sufficient notation.

share|cite|improve this answer
For the last example, it would be good to mention that $I(X,Y;Z) \neq I(X;Y,Z)$, so writing e.g. $I(X,Y,Z)$ would not be well-defined. – TMM Feb 22 '14 at 22:15

You often have $I(X_1,X_2,\ldots,X_n ; Y_1,Y_2, \ldots, Y_m)$. This is the mutual information between the vectors $(X_1,\ldots,X_n)$ and $(Y_1,\ldots,Y_m)$. The comma is used for vectors, the semicolon for separating the things between which the mutual information you're looking at.

This notation isn't confusing, since $I(X;Y;Z)$ doesn't define well (This is a standard homework problem from Cover & Thomas's Elements of Information Theory).

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.