Measure Theoretic Understanding of $P(X=x|Y=y)$ 
*

*The conditional probability $P(A|B)$ is defined to be $P(AB)/P(B)$, if $P(B)>0$.

*The conditional probability $P(A|\mathcal F)$ is defined to be $E(1_A|\mathcal F)$, if $\mathcal F$ is a sigma-field.

*Let $X,Y$ be jointly continuous random variables with joint density. We know there is a formula to compute
$$
P(X=x|Y=y),
$$
but since $P(Y=y)=0$, how could we understand it in a measure theoretic way?

 A: There is no need to assume $X$ and $Y$ are jointly continuous.  
First, observe that if $A$ is any Borel set, then $\mathbb{P}\{X \in A \, \mid \, \sigma(Y)\}$ is a $\sigma(Y)$-measurable random variable.  Therefore, I ask you to consider the following exercise: if $X_{1}$ is a $\sigma(Y)$-measurable random variable, then there exists a Borel measurable function $g : \mathbb{R} \to \mathbb{R}$ such that $X = g(Y)$.  In other words, $X$ is completely determined by the value of $Y$.  In our case, we learn that there is a function $g_{A}$ such that
$$\mathbb{P}\{X \in A \, \mid \, \sigma(Y)\}(\omega) = g_{A}(Y(\omega)).$$
Define $\mathbb{P}\{X \in A \, \mid \, Y = y\} = g_{A}(y)$.  Now observe that if $B$ is a Borel set, then
\begin{align*}
\mathbb{P}\{X \in A, Y \in B\} &= \int_{Y^{-1}(B)} \mathbb{P}\{X \in A \, \mid \, \sigma(Y)\}(\omega) \, \mathbb{P}(d \omega) \\
&= \int_{\Omega} g_{A}(Y(\omega)) \chi_{B}(Y(\omega)) \, \mathbb{P}(d \omega) \\
&= \int_{-\infty}^{\infty} g_{A}(y) \chi_{B}(y) \, \mathbb{P}_{Y}(dy) \\
&= \int_{B} \mathbb{P}\{X \in A \, \mid \, Y = y\} \, \mathbb{P}_{Y}(dy),
\end{align*}
where $\mathbb{P}_{Y} = Y_{*}(\mathbb{P})$ denotes the law of $Y$.
This tells us how to define $\mathbb{P}\{X \in A \, \mid \, Y = y\}$ in a way that agrees with the elementary definition.
A: Using the Lebesgue differentiation theorem, a quite elementary description is that
$$
P(x\in A|Y=y)=\lim_{\delta\to0}\frac{P(x\in A,Y\in B_\delta(y))}{P(Y\in B_\delta(y))}
$$
for almost every $y$, where $B_\delta(y)=(y-\delta,y+\delta)$.
