Joint distribution expressed with conditional distribution. Let $(\Omega,\mathfrak{A},\mathbb{P})$ be a probability space, $(\Omega',\mathfrak{A}')$, $(\Omega'',\mathfrak{A}'')$ two measure spaces and $$X\colon(\Omega,\mathfrak{A},\mathbb{P})\rightarrow (\Omega',\mathfrak{A}')$$ and $$Y\colon(\Omega,\mathfrak{A},\mathbb{P})\rightarrow (\Omega'',\mathfrak{A}'')$$ two random variables. Suppose  the conditional distribution $$\mathbb{P}(X\in\cdot|Y)$$ does exist.
How can one express the joint distribution $\mathbb{P}((X,Y)\in\cdot)$ in terms of the distributions of $X$, $Y$ and the conditional distribution?
Does $$\mathbb{P}((X,Y)\in\cdot)=\int_\Omega\int_{\Omega'}\delta_{(z,Y(\omega))}(\cdot)\mathbb{P}(X\in\cdot|Y)(\omega)(dz)\mathbb{P}(d\omega)$$ always hold? If so, how can one prove it?
 A: Your guess is true.
Since finite measures (especially probability measures) on the product of two  measure spaces are uniquely determined by their values on measurable rectangles and both side of the equation you guessed define probability measures on such a product space, it suffices to check the equality for measurable sets of the shape $A\times B\in \mathfrak{A}\otimes\mathfrak{B}$.
For those it holds
$$\mathbb{P}((X,Y)\in A\times B)\\=\mathbb{E}[\mathbb{1}_{(X,Y)\in A\times B}]\\=\mathbb{E}[\mathbb{E}[\mathbb{1}_{(X,Y)\in A\times B}|Y]]\\=\mathbb{E}[\mathbb{E}[\mathbb{1}_{X\in A}\mathbb{1}_{Y\in B}|Y]\\=\mathbb{E}[\mathbb{1}_{Y\in B}\mathbb{E}[\mathbb{1}_{X\in A}|Y]]\\=\mathbb{E}[\mathbb{1}_{Y\in B}\mathbb{P}(X\in A|Y)]\\={\int_\Omega\mathbb{1}_{Y\in B}(\omega)\int_{\Omega'}\mathbb{1}_{A}(z)\mathbb{P}(X\in \cdot|Y)(\omega)(dz)\mathbb{P}(d\omega)}\\= {\int_\Omega\int_{\Omega'}\delta_{(z,Y(\omega))}(A\times B)\mathbb{P}(X\in\cdot|Y)(\omega)(dz)\mathbb{P}(d\omega)},$$ where the fourth equality follows from one of the Kolmogorov axioms for the conditional expectation.
