I have a universe, $ U = \{a, b, c, d, e, f\}$ and sets $A = \{a, b, c\}$ and $B = \{a, d, e, f\}$
If $P(A) = P(X = x \in A)$ and $P(B) = P(X = x \in B)$, where $X$ is a random variable defined by uniformly selecting elements of $U$.
I have the following probabilities based on the above.
$ p(A) = 1/2$
$p(A,B) = 1/6$
$p(A|B) = 1/4$
I know that entropy $H(X) = - \sum(P(x_i) \log P(x_i))$ and information gain is $IG(X|Y) = H(X) - H(X|Y)$
Calculating entropy in A, I get: $H(A) = - 3(1/6) \log (1/6) = 0.389$
Now I am having a hard time to compute $IG (A | B)$ (which is defined as $H(A) - H(A|B)$) as I don't know how to compute the $H(A|B)$ here. Any clue?