1
$\begingroup$

let us suppose we have following joint probability distribution table enter image description here

i would like to calculate mutual information between two variable, first of all i have calculated marginal distributions enter image description here

and i used following formula enter image description here

which i have considered as this one enter image description here

Using excel I have calculated following table
enter image description here

zero is written because it is accepted generally that

$0*log(0)=0$

so finally i got following result

0.334497797

because it is different from zero, that means that these two random variable are not independent right?

$\endgroup$
2
$\begingroup$

It's all right.

because it is different from zero, that means that these two random variable are not independent right?

Yes. That can also be seen by checking that the joint probability doesn't equal the product of the marginals (at least for some values). For example $P(X=0,Y=0)=0.2$ while $P(X=0)P(Y=0)= 0.5 \times 0.3 = 0.15$

$\endgroup$
  • $\begingroup$ thanks for reply, what about if we dont have any information about their joint distribution? how can we check their independence? $\endgroup$ – dato datuashvili May 26 '17 at 21:15
  • 1
    $\begingroup$ @datodatuashvili If you only have the marginals, you can't know if they are independent. The joint distribution is what fully characterizes a pair of random variables. For checking non-independence , in some cases a less complete description might suffice: for example, non-zero covariance implies non-independent variables (but not the reverse). Also, if the "support" region (region of the domain where the joint probability function is non-zero) is not a rectangle (perhaps unbounded) or a cartesian product of rectangles, then they are not independent. $\endgroup$ – leonbloy May 26 '17 at 21:23
  • $\begingroup$ wish you good night thanks for your reply $\endgroup$ – dato datuashvili May 26 '17 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.