Multiple conditional variables in probability

I have trouble understanding, why my solution is wrong.

Given:

$P(A) = 0.5\\ \forall i: P(X_i |A ) = 0.2\\ \quad P(X_i|\neg A) = 0.6$

(And all $X_i$ are dependant on and only on $A$.) This is bad mathematical formulation. There was a graph, with $A$ at top and arrows from $A$ to all $X_i$

Asked: $P(A|X_1 X_2 \neg X_3)$

The solution should be $\approx 0.1818...$ As far as i know my numerator is correct and i don't understand why my denominator isn't.

My Approach:

I use the equation $P(A|B) = P(A B)/P(B)$ over and over.

$P(A|X_1 X_2 \neg X_3) = P(A X_1 X_2 \neg X_3)/P(X_1 X_2 \neg X_3) = \\ \frac{P(\neg X_3 | A X_1 X_2) P(AX_1X_2)}{P(X_1 X_2 \neg X_3)} = \frac{P(\neg X_3 | A X_1 X_2) P(X_2|AX_1) P(AX_1)}{P(X_1 X_2 \neg X_3)} =\\ \frac{P(\neg X_3 | A X_1 X_2) P(X_2|AX_1) P(X_1|A) P(A)}{P(X_1 X_2 \neg X_3)}$

Since $X_i$ is never dependant on any other $X_j$ i can simplify into:

$\frac{P(\neg X_3 | A) P(X_2|A) P(X_1|A) P(A)}{P(X_1 X_2 \neg X_3)}$

All the numbers for the numerator are given: $\frac{0.8\cdot 0.2 \cdot 0.2 \cdot 0.5}{P(X_1 X_2 \neg X_3)} = \frac{0.016}{P(X_1 X_2 \neg X_3)}$

So now i need to find out what $P(X_i)$ is. $P(X_i) = P(X_i|A) P(A) + P(X_i| \neg A) P(\neg A)\\ = 0.2 \cdot 0.5 + 0.6 \cdot 0.5 = 0.4$

Since $X_i$ are indpenendent, $P(X_1 X_2 \neg X_3) = P(X_1) P(X_2) P(\neg X_3)\\ = 0.4 \cdot 0.4 \cdot 0.6 = 0.096$

So $P(A|X_1 X_2 \neg X_3) = \frac{0.016}{P(X_1 X_2 \neg X_3)} = \frac{0.016}{0.096} = 0.166...$ which is not equal to $0.1818...$

What is my mistake with the denominator? Have i expanded incorrectly? Is my understanding of dependant/independant variables wrong?

(I know, no homeworks. This is from a udacity course. Complete solution is available. I just want to know, where i have done a mistake with my approach, because clearly i have difficulties with understanding dependent variables)

Official solution:

Get the numerator: $P'(A|X_1 X_2 \neg X_3) = 0.016$ (as with my solution)

Get the numerator: $P'(\neg A|X_1 X_2 \neg X_3) = 0.072$ (double checking with $= 0.4 \cdot 0.6 \cdot 0.6 \cdot 0.5$)

Both P's are not weighted and together they should add up to 1. So $0.016 + 0.072 = 0.088$ is the weighting factor.

$\Rightarrow \frac{P'(A|X_1 X_2 \neg X_3)}{P'(A|X_1 X_2 \neg X_3) + P'(\neg A|X_1 X_2 \neg X_3)} = \frac{0.016}{0.016+0.072} = \frac{0.016}{0.088} = 0.1818...$

• Your denominator is correct but your numerator is not. $P(\neg X_3|A)\neq 0.6$. – Mike Earnest Jun 5 '18 at 17:19
• that was just a typo, on my paper it was 0.8. The numerator is still 0.016 – hr0m Jun 5 '18 at 19:00

1 Answer

Your statements about independence are somewhat lax, and this is also what seems to have caused the error – I think this is a good opportunity to become more clear about what (in)dependence means and doesn't mean.

You write "All $X_i$ are dependent on and only on $A$", and you later conclude that the $X_i$ are independent. It's not entirely clear to me what exactly "being dependent on and only on $A$" means, but from the official solution, it seems that the intended meaning is: The $X_i$ are random variables whose joint distribution is as if you first throw a coin (event $A$) and then independently choose the $X_i$ depending on $A$, with the given conditional probabilities. This does not mean that the $X_i$ are independent; in fact, the fact that they all depend on the same event $A$ makes them dependent.

• That's exactly what i thought my mistake was. However i am very confused. So if they are dependent then $P(X_1 X_2 X_3) \neq P(X_1)P(X_2)P(X_3)$ However, why can i simplify $P(\neg X_3 | A X_1 X_2) = P(\neg X_3 |A)$? – hr0m Jun 5 '18 at 19:19
• All this juggling around the $|$ is really confusing :) – hr0m Jun 5 '18 at 19:21
• @hr0m: $P(\neg X_3 \mid A X_1 X_2) = P(\neg X_3 \mid A)$ (note that the spacing looks better if you use \mid instead of the vertical bar) because the $X_i$ "depend on each other only through $A$" -- if you know $A$, you know all there is to know about the distribution of $X_3$, there's no additional information in $X_1$ and $X_2$. But if you don't know $A$, then there is information in $X_1$ and $X_2$ about $X_3$ (since there's information about $A$), so they're not independent, just independent conditional on $A$ (see en.wikipedia.org/wiki/Conditional_independence). – joriki Jun 5 '18 at 19:45
• @hr0m: I suspect you mean $P(X_1X_2\neg X_3)$, since that's what you need in your calculation. You can compute either of these just like you computed $P(X_i)$: \begin{eqnarray*} P(X_1X_2\neg X_3)&=&P(X_1X_2\neg X_3\mid A)P(A)+P(X_1X_2\neg X_3\mid\neg A)\\ &=&0.2\cdot0.2\cdot0.8\cdot0.5+0.6\cdot0.6\cdot0.4\cdot0.5 \\&=&0.088\;. \end{eqnarray*} (That the probabilities conditional on $A$ or $\neg A$ are products follows from conditional independence with respect to $A$.) Using this in your calculation yields the right result: $$\frac{0.016}{0.088}=\frac2{11}=0.\overline{18}\;.$$ – joriki Jun 5 '18 at 21:02
• Aaaah of course. Thank you! It is so obvious now! – hr0m Jun 6 '18 at 5:50