If $X$ is discrete and $Z,W$ are discrete or continuous, is it always the case that $P(X=x\mid Z=z) \geq P(X=x\mid Z=z,W=w)$ for all $x,w,z$? Suppose $X$ is discrete and $Z,W$ are discrete or continuous, I am wondering if it is always the case (or at least non-trivially) that
$$
P(X=x\mid Z=z) \geq P(X=x\mid Z=z,W=w)
$$
for all $x\in X$, $w\in W$, and $z\in Z$?
It appears to make intuitive sense to me because it appears the probability of $X$ given $Z$ and $W$ should be "subsetting" off the probability of $X$ given $Z$. That is, if there is some chance of $X$ given $Z$, then $X$ given $Z$ and $W$ is subdividing the occurrence of $X$ given $Z$ into many more chunks according to $W$, and thus the probability of a chunk occurring should be less than the whole.
In other words, it seems to hold in the finite-sampling perspective, where the above are empirical proportions. However, I am unable to prove this generally. Is there a general result behind this?
 A: Certainly not true in general (in fact in 'most' cases). For example suppose $W=X$ and $X$ is independent of $Z$. If you take $w=x$ then the inequality becomes $P(X=x)^{2} \geq P(X=x)$ so it is false if $0<P(X=x) <1$.
A: The claimed relation is not true in general. As a counterexample, suppose $X$ and $W$ are independent and Bernoulli distributed random variables, with each of them taking the values $0$ and $1$ with equal probability. Further, suppose that
\begin{equation}
Z=X+W\quad \text{mod }2.
\end{equation}
Then, it is easy to see that $P(X=0|Z=0,W=0)=1$, whereas $P(X=0|Z=0)=\frac{1}{2}$, and therefore we have $P(X=0|Z=0)\ngeq P(X=0|Z=0,W=0)$.
It seems like you want to capture the fact that "given information about $Z$ and $W$, there is less uncertainty about the value that the random variable $X$ takes, than when given information only about $Z$" (this is true in the example I have mentioned above). This notion of uncertainty reducing upon conditioning more and more random variables is captured very nicely in the subject of information theory through a quantity known as Entropy. Please have a look at this quantity by visiting the link if you have not encountered it before.
So, in the language of information theory, it is true that
\begin{equation}
H(X|Z)\geq H(X|Z,W),
\end{equation}
where $H(X|Z)$ and $H(X|Z,W)$ denote the conditional entropy of $X$ given $Z$ and the conditional entropy of $X$ given $Z$ and $W$. In fact, the above relation between the conditional entropies is a consequence of a more general property: the difference between $H(X|Z)$ and $H(X|Z,W)$ is a quantity in information theory known as conditional mutual information of $X$ and $W$ given $Z$, which is denoted by $I(X;W|Z)$, and it is a known fact in information theory that  conditional mutual information is always nonnegative.
Hope this clarifies your doubt.
