# calculate mutual information using joint probability distribution table

let us suppose we have following joint probability distribution table

i would like to calculate mutual information between two variable, first of all i have calculated marginal distributions

and i used following formula

which i have considered as this one

Using excel I have calculated following table

zero is written because it is accepted generally that

$0*log(0)=0$

so finally i got following result

0.334497797


because it is different from zero, that means that these two random variable are not independent right?

## 1 Answer

It's all right.

because it is different from zero, that means that these two random variable are not independent right?

Yes. That can also be seen by checking that the joint probability doesn't equal the product of the marginals (at least for some values). For example $P(X=0,Y=0)=0.2$ while $P(X=0)P(Y=0)= 0.5 \times 0.3 = 0.15$

• thanks for reply, what about if we dont have any information about their joint distribution? how can we check their independence? – dato datuashvili May 26 '17 at 21:15
• @datodatuashvili If you only have the marginals, you can't know if they are independent. The joint distribution is what fully characterizes a pair of random variables. For checking non-independence , in some cases a less complete description might suffice: for example, non-zero covariance implies non-independent variables (but not the reverse). Also, if the "support" region (region of the domain where the joint probability function is non-zero) is not a rectangle (perhaps unbounded) or a cartesian product of rectangles, then they are not independent. – leonbloy May 26 '17 at 21:23
• wish you good night thanks for your reply – dato datuashvili May 26 '17 at 21:51