Bayesian Spam Classification Say I have 1000 e-mails in my inbox. I count the following things


*

*Spam 600, Ham 400

*Among Spam Mails: 100 from known senders, 90 contain the word 'credit'.

*Among Ham Mails: 200 from known senders, 10 contain the word 'credit'.


So there are 300 mails from known senders and 100 mails that contain the word 'credit'. 
I want to calculate P(Spam|Know & Credit), the probability that a mail is spam given that it comes from a known sender and contains the word 'credit'. By Bayes
$$P(S\,|\,K\cap C) = P(S) \frac{P(K\cap C\,|\,S)}{P(K \cap C)}$$
$P(S)=6/10$, and, since I assume independence $P(K\cap C\,|\,S)=P(K\,|\,S)\cdot P(C\,|\,S)$. Since there are 90 spams containing 'credit', and 100 spams from known senders, I have
$$P(K\cap C\,|\,S) = 100/600 \cdot 90/600 = 1/40$$
Now here is where I'm confused:
I assume independence, so I thought $P(K\cap C)=P(K)\cdot P(C )=300/1000\cdot 100/1000=3/100$. However, equally valid should be by the law of total probability 
$$P(K\cap C)=P(K\cap C\,|\, S) P(S)+ P(K\cap C\,|\,H)\cdot P(H)$$
and since things are independent I can pull them apart
$$P(K\cap C)=P(K\,|\, S) P(C\,|\, S) P(S)+P(K\,|\, H)P(C\,|\, H)\cdot P(H)$$
Plugging in the values I get
$$P(K\cap C) = 100/600 \cdot 90/600 \cdot 600/1000 + 200/400 \cdot 10/400 \cdot 400/1000 = 1/50$$
Only when I use 1/50 the overall answer makes sense, i.e. I get $P(S\,|\, K\cap C)=1-P(H\,|\, K\cap C)$. Why?
 A: Why should independance be justified? If among your acquaintances is your bank account manager, then the occurance of "credit" in mail from known sender may well be above arearage!
More specifically, you assume that $K$ and $C$ are independant three times:


*

*"In general", i.e. $P(K\cap C)=P(K)P(C)$;

*in case of spam, i.e. $P(K\cap C\mid S)=P(K\mid S)P(C\mid S)$;

*and in case of ham, i.e. $P(K\cap C\mid H)=P(K\mid H)P(C\mid H)$.


You cannot expect to have all three if $K$ and $C$ both are indicators (in the positive or negative) of spam vs. ham, i.e. correlated with $S$ (except for a specific overall probability of $S$, which apparently does not match the observed values).
A: The assumption that Naive Bayes classifier makes is not an independence assumption, it is a conditional independence assumption given the class. So in your context it only makes sense to say:
$P(K \cap C \mid S)=P(K \mid S)P(C\mid S)$, and
$P(K \cap C \mid H)=P(K \mid H)P(C\mid H)$.
Since we are not making the independence assumption (also known as unconditional independence or mutual independence), we need another way of estimating $P(K \cap S)$ and this is where the law of total probability comes in handy.

Although this is outside the scope of your question, it might be interesting to reason about the conditional independence assumption was made in the first place. If we hadn't made such assumption, we would need to directly calculate $P(K \cap C \mid S)$ and similar values, and that would mean we would need to get samples for all feature combinations: (known sender, "credit" present), (unknown sender, "credit" present), and so on. And normally we don't have 2 features, but more in the order of 10,000s. And this means $2^{10,000}$ samples, which is madness.

Note: the previous answer does lead in the right direction, but it leaves one confused about exactly what assumption Naive Bayes classifier makes.
