2
$\begingroup$

If $f(x,y)$ is a joint pdf,I understand that,

$$ \int_{-\infty}^\infty\int_{-\infty}^\infty f(x,y) \ dx \ dy = 1 $$

but does this hold for the conditional pdf?

$$ \int_{-\infty}^\infty f({y|x}) \ dy = 1. $$

I would argue not. For a given $X=x$ the sum of of the pdf over the support of $Y$ should less than or equal to 1. But according to Bierens it is.

In particular, in the case (3.4) we have \begin{align} \begin{split} E[(Y &- E[Y \mid X]) I(X \in B)] \\ &= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (y - g(x)) I(x \in B) f(y, x) \ dy \ dx \\ &= \int_{-\infty}^{\infty} \left( \int_{-\infty}^{\infty} y f(y \mid x) \ dy \right) I(x \in B) f_x(x) \ dx \\ &\quad - \int_{-\infty}^{\infty} \left( \int_{-\infty}^{\infty} f(y \mid x) \ dy \right) g(x) I (x \in B) f_x(x) \ dx \\ &= \int_{-\infty}^{\infty} g(x) I(x \in B) f_x(x) \ dx - \int_{-\infty}^{\infty} g(x) I (x \in B) f_x(x) \ d x = 0. \end{split} \tag{3.7} \end{align}

$\endgroup$
  • $\begingroup$ Not clear your question and your last expression, that is not the expression for the conditional expectation, is just the integral over a conditional PDF. $\endgroup$ – Carlos Mendoza Dec 6 '15 at 3:46
  • $\begingroup$ Sorry it was a typo. I meant over the conditional pdf. Does it sum to 1 over the support of y given a fixed x? $\endgroup$ – jessica Dec 6 '15 at 3:48
  • $\begingroup$ It does. Take a look at the answers and tell us what you think after that. $\endgroup$ – Carlos Mendoza Dec 6 '15 at 4:22
  • $\begingroup$ $f_{Y|X}(y|x)$ for any fixed $x$ should be a valid pdf - so it should integrate to $1$. $\endgroup$ – A.S. Dec 6 '15 at 4:36
  • $\begingroup$ I added some graphs and extra explanation to make this more clear, hopefully ;) $\endgroup$ – Carlos Mendoza Dec 6 '15 at 6:24
4
$\begingroup$

By definition,

$$f_{Y \mid X}(y \mid x) = \frac{f_{X,Y}(x,y)}{f_X(x)}\qquad \forall x,y$$

Here, given $x$, $f_X(x)$ is a constant (we are evaluating at some $x$). Then,

\begin{align} \int_{-\infty}^{\infty} f_{Y \mid X}(y \mid x) dy &= \frac{\int_{-\infty}^\infty f_{X,Y}(x,y)dy}{f_X(x)}\\ &= \frac{f_X(x)}{f_X(x)}\\ &= 1\qquad \forall x \end{align}

where in the second equality we have computed the marginal PDF of $X$ by integrating the joint PDF over $y$.

Let's see now a illustrated example to hopefully gain a better understanding about this result. Suppose that $f_{X,Y}(x,y)$ is as shown below (in red a "view" of the joint PDF at $Y=2.5$)

$\hspace{2.5cm}$ enter image description here

Suppose we are interested in $f_{X \mid Y}(x \mid 2.5)$, which is

$$f_{X \mid Y}(x \mid 2.5) = \frac{f_{X,Y}(x,2.5)}{f_Y(2.5)} = \frac{1/4}{2(1/4)} = \frac{1}{2}\qquad 1\leq x \leq 3$$

Note that $f_{X \mid Y}(x \mid 2.5)$ has the same shape than $f_{X,Y}(x,2.5)$ (a constant equal to $0.25$), but it's divided by $f_Y(2.5)$ to normalize it. That is, to satisfy the normalization axiom! So, although the area under $f_{X,Y}(x, 2.5)$ (the red area above) is clearly smaller than $1$, the area under $f_{X \mid Y}(x \mid 2.5)$ is $1$ because of the mentioned normalization. Below the graph for this conditional PDF.

$\hspace{2cm}$ enter image description here

EDIT: In response to your comment, Jessica. You are right about the intuition. I just want to add something to it using the example above. We are working with two random variables, but both of them are defined over the same sample space $\Omega$, which would be represented by that T-shape region in the $x-y$ plane. The probability law defined over $\Omega$ is $f_{X,Y}$, and the volume under it is $1$. Now, what is $f_{X \mid Y}$? It is a new probability law defined over a new sample space, the one that results of imposing the restriction $Y=2.5$ to $\Omega$, that is $\{1 \leq X \leq 3, Y=2.5$}. A key idea here is that when we condition the original model the shape of the original probability law, $f_{X,Y}$, over that new sample space does not change, is just scaled, and that scaling is what help us to hold the normalization axiom. We have reduced the sample space but at the same time we have increased the height of the distribution. It is a beautiful intuitive idea that is right there in the definition of a conditional probability: we start we a given sample space $\Omega$ and a probability law $P(\cdot)$ that enable us to compute something like $P(A)$, $P(B)$ or even $P(A \cap B)$. But what if we know something, for example that $A$ has occurred? Then we think intuitively that our new sample space is $A$. How we compute now $P(B)$? We represent that probability now as $P(B \mid A)$ to incorporate in the notation that partial knowledge about the result and realize that $B$ occurs only if $A \cap B$ occurs to computed it as

$$P(B \mid A) = \frac{P(A \cap B)}{P(B)}$$

why we divide by $P(B)$? To normalize the non-conditional probabilities so that the conditional ones add up to $1$! Now that our sample space got reduced, is reasonable to increase proportionally the probabilities in the new sample space. This is one of my favorites concepts in probability, the conditional probability.

$\endgroup$
2
$\begingroup$

Yes, it does $$\int_{-\infty}^\infty f_{Y|X}(y|x)\,dy = \int_{-\infty}^\infty\frac{f_{X,Y}(x,y)}{f_X(x)}dy =\frac{1}{f_X(x)}\cdot f_X(x) = 1.$$

As an example, if $Y|X\sim \text{Exp}(X)$, then $$\int_{-\infty}^\infty f_{Y|X}(y|x)\,dy = \int_0^\infty xe^{-xy}dy = 1.$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.