1
$\begingroup$

I have problem in proving $H(Y \mid X) \leq H(Y \mid f(X))$, where $f$ is a function of $X$. In the textbooks, they already proved $I(Y; X) \geq I(Y;f(X))$, and $H(f(X)) \leq H(X)$ but I can't relate those with the question problem.

Besides, there is one other question that I am concerned about. If $H(Y \mid X) = H(Y \mid f(X))$, what is the conditions? Is that $p_Y(\cdot \mid X) = p_Y(\cdot \mid f(X))$?

We already have the case $H(Z) = H(W) \rightarrow p_Z(\cdot) = p_W(\cdot)$, that is wrong. However I doubt it may be right for $H(Y \mid X) = H(Y \mid f(X)) \rightarrow p_Y(\cdot \mid X) = p_Y(\cdot \mid f(X))$.

Can anyone help? Thanks.

$\endgroup$
6
  • $\begingroup$ I can help you if you promise to never ever again use the incorrect notation $P(Z)$ and $P(W)$, $P(Y|X)$, $P(Y|f(X)$. You can only take probabilities of events, and the same for conditional probabilities. You can use a mass function $p_Z(z) =P[Z=z]$ for all $z$ in the set of possible outcomes of the random variable $Z$. You can also use conditional probabilities $P[Y=y|X=x]$. $\endgroup$
    – Michael
    Jul 29, 2018 at 4:44
  • $\begingroup$ Some textbooks use different notations, so it causes confusion. However, I understand each notation means. $P(X)$ in my writings means $p_X(x)$. In this case, sorry for what cause troubles reading to you. Please rectify any mistakes in my notation, and help me prove it. So much thanks. I have revised the equations as you suggested. $\endgroup$
    – khahuras
    Jul 29, 2018 at 5:37
  • 2
    $\begingroup$ Hint: it holds $I(X;Y)\geq I(Y;f(X))$ and also $I(X;Y)=H(Y)-H(Y|X)$. $\endgroup$
    – Stelios
    Jul 29, 2018 at 6:51
  • $\begingroup$ Thanks Stelios! $\endgroup$
    – khahuras
    Jul 29, 2018 at 11:05
  • 1
    $\begingroup$ The Stelios comment is what I was going to give as a hint also. As for when $H(X|f(Y))=H(X|Y)$ holds, intuitively it is when $f(Y)$ tells you just as much "information" about $X$ as $Y$ tells you about $X$, so $I(X; Y)=I(X;f(Y))$. It holds under various cases, such as when $X$ and $Y$ are independent, or when $f$ is invertible (so knowledge of $f(Y)$ tells you $Y$). $\endgroup$
    – Michael
    Aug 2, 2018 at 17:26

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .