# If $X$ is discrete and $Z,W$ are discrete or continuous, is it always the case that $P(X=x\mid Z=z) \geq P(X=x\mid Z=z,W=w)$ for all $x,w,z$?

Suppose $$X$$ is discrete and $$Z,W$$ are discrete or continuous, I am wondering if it is always the case (or at least non-trivially) that

$$P(X=x\mid Z=z) \geq P(X=x\mid Z=z,W=w)$$

for all $$x\in X$$, $$w\in W$$, and $$z\in Z$$?

It appears to make intuitive sense to me because it appears the probability of $$X$$ given $$Z$$ and $$W$$ should be "subsetting" off the probability of $$X$$ given $$Z$$. That is, if there is some chance of $$X$$ given $$Z$$, then $$X$$ given $$Z$$ and $$W$$ is subdividing the occurrence of $$X$$ given $$Z$$ into many more chunks according to $$W$$, and thus the probability of a chunk occurring should be less than the whole.

In other words, it seems to hold in the finite-sampling perspective, where the above are empirical proportions. However, I am unable to prove this generally. Is there a general result behind this?

• $\mathbb{P}(X)$ is meaningless when $X$ is a random variable. E.g., if $X$ is the result of a die roll (a number from 1 to 6), $\mathbb{P}(X)$ is "the probability of the result of a die roll" (meaningless). Is $X$ an event? If so, what does it mean for an event to be discrete? – parsiad May 14 at 4:19
• Are you implying I should fix $X=x$. I have changed the above. – user321627 May 14 at 4:21
• This seems like the basic idea of conditional probability (en.wikipedia.org/wiki/Conditional_probability) . So in the post you did not mention whether $X, Y, Z$ are independent or not. In the case where the variables are independent, then $Z$ and $W$ contribute no information about $X$, no matter whether any of the variables are discrete or continuous. Perhaps you can clarify the dependency structure you are assuming in the post itself. – krishnab May 14 at 4:26
• I am assuming that there is no independence existing either marginally nor conditionally among any of the variables. I was hoping for a general result. – user321627 May 14 at 4:28

Certainly not true in general (in fact in 'most' cases). For example suppose $$W=X$$ and $$X$$ is independent of $$Z$$. If you take $$w=x$$ then the inequality becomes $$P(X=x)^{2} \geq P(X=x)$$ so it is false if $$0.
The claimed relation is not true in general. As a counterexample, suppose $$X$$ and $$W$$ are independent and Bernoulli distributed random variables, with each of them taking the values $$0$$ and $$1$$ with equal probability. Further, suppose that $$\begin{equation} Z=X+W\quad \text{mod }2. \end{equation}$$ Then, it is easy to see that $$P(X=0|Z=0,W=0)=1$$, whereas $$P(X=0|Z=0)=\frac{1}{2}$$, and therefore we have $$P(X=0|Z=0)\ngeq P(X=0|Z=0,W=0)$$.
It seems like you want to capture the fact that "given information about $$Z$$ and $$W$$, there is less uncertainty about the value that the random variable $$X$$ takes, than when given information only about $$Z$$" (this is true in the example I have mentioned above). This notion of uncertainty reducing upon conditioning more and more random variables is captured very nicely in the subject of information theory through a quantity known as Entropy. Please have a look at this quantity by visiting the link if you have not encountered it before.
So, in the language of information theory, it is true that $$\begin{equation} H(X|Z)\geq H(X|Z,W), \end{equation}$$ where $$H(X|Z)$$ and $$H(X|Z,W)$$ denote the conditional entropy of $$X$$ given $$Z$$ and the conditional entropy of $$X$$ given $$Z$$ and $$W$$. In fact, the above relation between the conditional entropies is a consequence of a more general property: the difference between $$H(X|Z)$$ and $$H(X|Z,W)$$ is a quantity in information theory known as conditional mutual information of $$X$$ and $$W$$ given $$Z$$, which is denoted by $$I(X;W|Z)$$, and it is a known fact in information theory that conditional mutual information is always nonnegative.