# Tag Info

1

Usually one requires certain properties (axioms) for an entropy: Nonnegativity; the bigger the uncertainty, the bigger the entropy; and additivity for independent observations/measurements. The last property implies that there should be a logarithmic dependence. For more details on the axiomatic formulation of entropy see ...

0

You are simply looking for the solutions of the implicit equation: $x \log (x)+y\log (y)+(1-x-y)\log (1-x-y) = 0.2\log (0.2)+0.3\log (0.3)+ 0.5 \log (0.5)$ For example, you can try a plot here

1

There are such pmf coincidences, applicable for any set of candidates with more than 2 candidate values other than the maximal entropy all-value-equal entropy. For your example, consider (although there is an entire 1-parameter family of iso-entropic distributions) the distrbution with $$p(X) = [0.24301892,0.24301892,0.51396216]$$ This has the identical ...

2

From our definition of conditional entropy: $$H(Y|X) = \sum_{x}p_{X}(x)H(Y|X=x) = -\mathbb{E}[\text{log}p(Y|X)]$$ We know the chain rule which states that: $$H(X,Y) = H(X) + H(Y|X) = H(Y) + H(X|Y)$$ so $$H(Y|X) - H(X|Y) = H(Y) - H(X) = 0$$ by the fact that X and Y have the same distribution. Of course this is clear from the symmetry.

Top 50 recent answers are included