Integral Over Conditional PDF

Question

If $f(x,y)$ is a joint pdf,I understand that,

$$ \int_{-\infty}^\infty\int_{-\infty}^\infty f(x,y) \ dx \ dy = 1 $$

but does this hold for the conditional pdf?

$$ \int_{-\infty}^\infty f({y|x}) \ dy = 1. $$

I would argue not. For a given $X=x$ the sum of of the pdf over the support of $Y$ should less than or equal to 1. But according to Bierens it is.

In particular, in the case (3.4) we have \begin{align} \begin{split} E[(Y &- E[Y \mid X]) I(X \in B)] \\ &= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (y - g(x)) I(x \in B) f(y, x) \ dy \ dx \\ &= \int_{-\infty}^{\infty} \left( \int_{-\infty}^{\infty} y f(y \mid x) \ dy \right) I(x \in B) f_x(x) \ dx \\ &\quad - \int_{-\infty}^{\infty} \left( \int_{-\infty}^{\infty} f(y \mid x) \ dy \right) g(x) I (x \in B) f_x(x) \ dx \\ &= \int_{-\infty}^{\infty} g(x) I(x \in B) f_x(x) \ dx - \int_{-\infty}^{\infty} g(x) I (x \in B) f_x(x) \ d x = 0. \end{split} \tag{3.7} \end{align}

Answer 1

By definition,

$$f_{Y \mid X}(y \mid x) = \frac{f_{X,Y}(x,y)}{f_X(x)}\qquad \forall x,y$$

Here, given $x$, $f_X(x)$ is a constant (we are evaluating at some $x$). Then,

\begin{align} \int_{-\infty}^{\infty} f_{Y \mid X}(y \mid x) dy &= \frac{\int_{-\infty}^\infty f_{X,Y}(x,y)dy}{f_X(x)}\\ &= \frac{f_X(x)}{f_X(x)}\\ &= 1\qquad \forall x \end{align}

where in the second equality we have computed the marginal PDF of $X$ by integrating the joint PDF over $y$.

Let's see now a illustrated example to hopefully gain a better understanding about this result. Suppose that $f_{X,Y}(x,y)$ is as shown below (in red a "view" of the joint PDF at $Y=2.5$)

$\hspace{2.5cm}$

Suppose we are interested in $f_{X \mid Y}(x \mid 2.5)$, which is

$$f_{X \mid Y}(x \mid 2.5) = \frac{f_{X,Y}(x,2.5)}{f_Y(2.5)} = \frac{1/4}{2(1/4)} = \frac{1}{2}\qquad 1\leq x \leq 3$$

Note that $f_{X \mid Y}(x \mid 2.5)$ has the same shape than $f_{X,Y}(x,2.5)$ (a constant equal to $0.25$), but it's divided by $f_Y(2.5)$ to normalize it. That is, to satisfy the normalization axiom! So, although the area under $f_{X,Y}(x, 2.5)$ (the red area above) is clearly smaller than $1$, the area under $f_{X \mid Y}(x \mid 2.5)$ is $1$ because of the mentioned normalization. Below the graph for this conditional PDF.

$\hspace{2cm}$

EDIT: In response to your comment, Jessica. You are right about the intuition. I just want to add something to it using the example above. We are working with two random variables, but both of them are defined over the same sample space $\Omega$, which would be represented by that T-shape region in the $x-y$ plane. The probability law defined over $\Omega$ is $f_{X,Y}$, and the volume under it is $1$. Now, what is $f_{X \mid Y}$? It is a new probability law defined over a new sample space, the one that results of imposing the restriction $Y=2.5$ to $\Omega$, that is $\{1 \leq X \leq 3, Y=2.5$}. A key idea here is that when we condition the original model the shape of the original probability law, $f_{X,Y}$, over that new sample space does not change, is just scaled, and that scaling is what help us to hold the normalization axiom. We have reduced the sample space but at the same time we have increased the height of the distribution. It is a beautiful intuitive idea that is right there in the definition of a conditional probability: we start we a given sample space $\Omega$ and a probability law $P(\cdot)$ that enable us to compute something like $P(A)$, $P(B)$ or even $P(A \cap B)$. But what if we know something, for example that $A$ has occurred? Then we think intuitively that our new sample space is $A$. How we compute now $P(B)$? We represent that probability now as $P(B \mid A)$ to incorporate in the notation that partial knowledge about the result and realize that $B$ occurs only if $A \cap B$ occurs to computed it as

$$P(B \mid A) = \frac{P(A \cap B)}{P(B)}$$

why we divide by $P(B)$? To normalize the non-conditional probabilities so that the conditional ones add up to $1$! Now that our sample space got reduced, is reasonable to increase proportionally the probabilities in the new sample space. This is one of my favorites concepts in probability, the conditional probability.

Answer 2

Yes, it does $$\int_{-\infty}^\infty f_{Y|X}(y|x)\,dy = \int_{-\infty}^\infty\frac{f_{X,Y}(x,y)}{f_X(x)}dy =\frac{1}{f_X(x)}\cdot f_X(x) = 1.$$

As an example, if $Y|X\sim \text{Exp}(X)$, then $$\int_{-\infty}^\infty f_{Y|X}(y|x)\,dy = \int_0^\infty xe^{-xy}dy = 1.$$