Summing over conditional probabilities

I've seen the following formula being used in various places:

$P(a|b) = \sum_z P(a|z) \times P(z|b)$

So essentially they sum over all the variables z to get the original conditional probability. Is this correct, and does it make any strong assumptions?

I am confused by the following example:

$P(good\ weather | month) = \\ \sum_{z \in \{T,F\}} P(good\ weather| universe\ exists = z) * P(universe\ exists = z| month) = \\ P(good\ weather|universe\ exists = T)$

It starts off with the probability of having good weather, given the current month. We then introduce an additional variable (does the universe exist), and sum over all values (T,F). Let's say that the universe exists with probability 1.0, then:

$P(universe\ exists = T| month) = 1.0$

$P(universe\ exists = F| month) = 0.0$

Therefore, we end up with:

$P(good\ weather | month) = P(good\ weather | universe\ exists = T)$

Now, somewhere I have clearly lost some important information, because I'm showing that the conditional probability of good weather, depending on the month, is equal to the probability of good weather, given that the universe exists. What am I missing?

-

1 Answer

The equation does certainly not apply in general. What you can write however is $$P(a|b) = \sum_z P(a,z|b),$$ which is sometimes referred to as marginalization. Note that the above equation simply describes how to go from a joint probability mass function $P(x,y)$ to the probability mass function $P(x)$ (or $P(y)$), that is, by summing out the other variable. (Similar arguments apply to probability density functions by replacing the above summation with integration.)

Now, one can apply Bayes rule and further expand $P(a,z|b) = P(a|z,b) P(z|b)$ to get$$P(a|b) = \sum_z P(a|z,b) P(z|b).$$ The formula that you posted simply drops the conditioning on $b$ in the first factor (implying $P(a|z,b) = P(a|z)$), which cannot be done in general.

-
Thanks, that's exactly what I was looking for. Although, I think in the last equation you meant $P(a|z,b)=P(a|z)$. – Marek Nov 14 '12 at 11:17
You are welcome. I edited the typo you mentioned. – TriSSSe Nov 14 '12 at 13:32