Binary classification, Bayes classifier, Bayes decision boundary I have recently come across this problem from a friend I help with stats occasionally. This however stumped me completely. I have looked online on basically every single website you can find but what I did find either I did not fully understand or I didn't fully understand well enough to explain to my friend. In my last resort I have made an account here hoping for some help. 
From what I have read and understood I know the Bayes boundary is a kind of squiggly line you find which separates between classifying an observation on either end. However I don't understand how one comes to finding how to get one. Furthermore, bayesian stats is very new to me so I am struggling and therefore my friend is too. Thank you for reading and I hope someone who understands this well can explain it to me in a simple concise and easy way 
Consider a binary classification problem $Y \in \{0, 1\}$ with one predictor $X$.
The prior probability of being in class 0 is $Pr(Y = 0) = \pi_0= 0.69$ and the density
function for $X$ in class 0 is a standard normal
$$f_0(x) = Normal(0, 1) = (1/\sqrt{2\pi})\exp(-0.5x^2).$$ 
The density function for $X$ in class 1 is also normal, but with $\mu = 1$ and $\sigma^2 = 0.5$, i.e.
$$f_1(x) = Normal(0, 1) = (1/\sqrt{\pi})\exp(-(x-1)^2).$$ 
(a) Plot $\pi_0f_0(x)$ and $\pi_1f_1(x)$ in the same figure.
(b) Find the Bayes decision boundary.
(c) Using Bayes classifier, classify the observation $X = 3$. Justify your prediction.
(d) What is the probability that an observation with $X = 2$ is in class 1?
Any help at all would be greatly appreciated!
 A: I think I'm interpreting this problem correctly, but I might also just be kidding myself.
Basic approach. Suppose that we observe $X = x$.  We are looking for, essentially, $P(Y = y \mid x < X < x+dx), y = 0, 1$.  Bayes permits us to rewrite this as
$$
P(Y = y \mid x < X < x+dx)
    = \frac{P(x < X < x+dx \mid Y = y) P(Y = y)}{P(x < X < x+dx)}
$$
We are given that
$$
\pi_y \stackrel{\text{def}}{=} P(Y = y) =\begin{cases}
    0.69 & y = 0 \\
    0.31 & y = 1
\end{cases}
$$
and conveniently,
$$
f_y(x)\,dx = P(x < X < x+dx \mid Y = y)
$$
I would guess that the decision boundary lies wherever
$$
P(Y = 0 \mid x < X < x+dx) = P(Y = 1 \mid x < X < x+dx) = \frac12
$$
On the basis of the foregoing, we can rewrite this as
$$
\pi_0 f_0(x) \, dx = \pi_1 f_1(x) \, dx
$$
or, equivalently,
$$
\pi_0 f_0(x) = \pi_1 f_1(x)
$$
For this problem, I make it that the boundary consists of two values of $x$; between these values, one of the values of $Y$ is preferred; outside of those values, the other value of $Y$ is preferred.
A: Given a predictor $X$, you want to know from which gaussian it was most likely sampled.
So you want to calculate $\mathbb{P} (Y = 0 | X)$ for instance. Using bayes rule, and the fact that we have access to $\mathbb{P} (X | Y = 0)$:
$$ \mathbb{P} (Y = 0 | X)= \frac{\mathbb{P} (X | Y = 0) \pi_0}{\mathbb{P}(X)} $$
and $\mathbb{P}(X) = \pi_0f_0(x) +  \pi_1f_1(x)$ by the law of total probability.
 In summary :
$$  \mathbb{P} (Y = 0 | X)= \frac{\pi_0 f_0(x)}{\pi_0f_0(x) +  \pi_1f_1(x)} $$
$$ \mathbb{P} (Y = 1 | X)= \frac{\pi_1 f_1(x)}{\pi_0f_0(x) +  \pi_1f_1(x)} $$
And you're going to predict $Y=0$ for $X$ if $$\mathbb{P} (Y = 0 | X) \gt \mathbb{P} (Y = 1 | X) $$
meaning :
$$ \pi_0 f_0(x) \gt \pi_1 f_1(x) $$
And the decision boundary is the $x$ solution to:  $\pi_0 f_0(x) = \pi_1 f_1(x)$.
I'll leave the calculations to you because it's pretty basic.
The intuition behind this is that : If you have two gaussians $G_1$ (red) and $G_2$ (blue) that have same probability $1/2$ (like in the figure below), you're going to predict $G_1$ for $X$ if $X \leq 0$, and $G_2$ otherwise. Here the decision boundary is the intersection between the two gaussians. In a more general case where the gaussians don't have the same probability and same variance, you're going to have a decision boundary that will obviously depend on the variances, the means and the probabilities. I suggest that you plot other examples to get more intuition.

