# Compute the posterior probability in a Naive Bayes classifier

Consider a binary classification with one binary output $$y$$ and two binary features $$x_1$$ and $$x_2$$. The Naive Bayes classifier assumes the following distribution for a pair: $$p(y,x_1, x_2) = p(x_1|y) p(x_2|y) p(y)$$ Let the values of the probabilities be \begin{align*} p(y = 0) &= 0.5\\ p(x_1 = 1|y = 0) &= 0.9 \\ p(x_2 = 1|y = 0) &= 0.5 \end{align*}

\begin{align*} p(y = 1) &= 0.5\\ p(x_1 = 1|y = 1) &= 0.2 \\ p(x_2 = 1|y = 1) &= 0.5 \end{align*} Assumption: the features $$x_i$$ are assumed to be conditionally independent given the target $$y$$. For example $$p(x_1|y,x_2) = p(x_1|y)$$ Compute the posterior values: $$p(y = 1|x_1 = 1, x_2 = 1) \quad \text{and} \quad p(y = 0|x_1 = 1, x_2 = 1)$$ How would you classify an exmple with $$x_1 = 1$$ and $$x_2 = 1$$ \newline \newline \textbf{solution}: Using Bayes rule we have: \begin{align*} p(y = 1|x_1 = 1, x_2 = 1) &= \frac{p(x_1 = 1, x_2 = 1|y = 1)p(y = 1)}{p(x_1 = 1, x_2 = 1)}\\ &\propto p(x_1 = 1, x_2 = 1|y = 1)p(y = 1)\\ &= 0.2\cdot 0.5 \cdot 0.5\\ &= 0.05\\ &\text{}\\ &\text{and}\\ &\text{}\\ p(y = 0|x_1 = 1, x_2 = 1) &= \frac{p(x_1 = 1, x_2 = 1|y = 0)p(y = 1)}{p(x_1 = 1, x_2 = 1)}\\ &\propto p(x_1 = 1, x_2 = 1|y = 0)p(y = 0)\\ &= 0.9\cdot 0.5 \cdot 0.5\\ &= 0.225 \end{align*} Therefore we get $$p(y = 1|x_1 = 1, x_2 = 1) = 0.2/1.1 \approx 0.18 \quad \textbf{(1)}$$ The example should be classified as $$y = 0$$

Here is what I don't understand: how do you get both $$0.2$$ and $$1.1$$ in (1) with the given information to be able to compute the result of (1)?

I do not know where those numbers come from but here is how I would do the computation. $$P(x_1=1,x_2=1|y=1)=P(x_1=1|y=1)P(x_2=1|y=1)=0.2\cdot 0.5=0.1$$ (by conditional independence). Same with $$y=0$$: $$P(x_1=1,x_2=1|y=0)=P(x_1=1|y=0)P(x_2=1|y=0)=0.9\cdot 0.5=0.45$$ Next,to be able to use Bayes rule, we need the total probability $$P(x_1=1,x_2=1)=P(x_1=1,x_2=1|y=1)P(y=1)+P(x_1=1,x_2=1|y=0)P(y=0)=0.1\cdot0.5+0.45\cdot 0.5=0.275$$ Using Bayes rule now, $$P(y=1|x_1=1,x_2=1)=\frac{P(x_1=1,x_2=1|y=1)P(y=1)}{P(x_1=1,x_2=1)}=\frac{0.1\cdot 0.5}{0.275}=0.18.$$