I'm trying to better understand the measure theoretic definition of a probability density function, but I believe I'm making a mistake somewhere if someone could clarify. From what I currently gather, let $(\Omega,F,\mathbb{P})$ be a probability space and let $X:\Omega\rightarrow \mathbb{R}$ be a real valued random variable. Then, the distribution of $X$ is the push-forward measure $$ X_*\mathbb{P}(E)=\mathbb{P}(X^{-1}(E))=\mathbb{P}(X\in E) $$ where $E\subseteq \mathbb{R}$ is some measurable set in $\mathbb{R}$. The cumulative distribution function of $X$ is then $$ F(a) = \mathbb{P}(X\in (-\infty,a]) = \mathbb{P}(X^{-1}(-\infty,a]) $$ To obtain the probability density function, we assume that $\mathbb{P}<<\lambda$ where $\lambda$ is the Lebesgue measure and then note $$ F(a)=\mathbb{P}(X^{-1}(-\infty,a]) = \int_{X^{-1}(-\infty,a]} 1 d\mathbb{P} = \int_{X^{-1}(-\infty,a]} \frac{d\mathbb{P}}{d \lambda} d\lambda $$ where $\frac{d\mathbb{P}}{d \lambda}$ is the Radon-Nikodym derivative of $\mathbb{P}$ with respect to $\lambda$. We then set $f=\frac{d\mathbb{P}}{d \lambda}$ and call $f$ the probability density function.

Alright, so I know there's an error here since I really want $$ F(a) = \int_{-\infty}^a f d\lambda $$ but instead I have $$ \int_{X^{-1}(-\infty,a]} f d\lambda $$ Basically, there's $X^{-1}$ here and I don't think there should be, so what's the error in what I'm doing with Radon-Nikodym? Second, I'm still not entirely sure how to define $\mathbb{P}$ concretely. Say we're working with a normal distribution, I know that $$ f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} $$ and $$ F(x) = \frac{1}{2}\left[1+\textrm{erf}\left(\frac{x-\mu}{\sigma\sqrt{2}}\right)\right] $$ In this case, what would $\mathbb{P}$ be?

Thanks for the help in advance.

  • $\begingroup$ This might be helpful: en.wikipedia.org/wiki/Probability_density_function $\endgroup$ – i707107 Oct 8 '16 at 5:17
  • 3
    $\begingroup$ Try to spot which measure is defined on which set: here, $X:\Omega\to\mathbb R$, $P$ is a measure on $\Omega$ and $\lambda$ is a measure on $\mathbb R$, hence $P\ll\lambda$ is absurd and should read $X_*P\ll\lambda$. The rest follows, for example, the PDF of $X$, when it exists, is $$f=\frac{d(X_*P)}{d\lambda}$$ which is equivalent to the fact that for every Borel subset of $\mathbb R$, $$P(X\in B)=\int_Bf\,d\lambda$$ $\endgroup$ – Did Oct 8 '16 at 8:17
  • $\begingroup$ @Did Makes sense and that helps a lot. Could you clarify the second part? Basically, for a normal distribution, how do we define the measure $P$ on $\Omega$? $\endgroup$ – wyer33 Oct 9 '16 at 4:41
  • $\begingroup$ Again: the point is that one does not need to "define P concretely" since P lives on the source set Omega while one works with the PDF and the CDF, that are defined on the image set. $\endgroup$ – Did Oct 9 '16 at 6:07
  • $\begingroup$ @Did Certainly, but if we wanted to define it, could we? Alternatively, am I going about this backwards? Meaning, start with $$ F(B)=\int_B f d\lambda $$ then, push $f$ into the measure $$ F(B)=\int_B 1 d\mu $$ using Radon-Nikodym where $(\mathbb{R},B,\mu)$. Then, for some random variable $X:\Omega\rightarrow\mathbb{R}$, there exists a probability measure $P$, $(\Omega,F,P)$, such that $\mu=X_*P$. We may not know what $P$ is and we may not need it, but one exists. $\endgroup$ – wyer33 Oct 9 '16 at 20:21

This amounts to proving that

$P[X\in B]=\int_BdF,$

where $B$ is a Borel set in $(-\infty, \infty)$ and $X$ is a random variable with distribution funtion $F$.

Let $\mu_F$ denote the Lebesgue-Stieltjes measure over $(-\infty,\infty)$ determined by $F$, and let's take $B$ of the form $B=\cup_{i=1}^n(a_i,b_i]$. Then we have

$\mu_F(B)=\sum_{i=1}^n(F(b_i)-F(a_i))=\sum_{i=1}^nP(a_i< X \leq b_i)=P(\cup_{i=1}^n\{a_i < X \leq b_i\})=P(X^{-1}B)$

$PX^{-1}$ is a measure since $\mu_F$ is, hence we can extend uniquely both measures to the sigma field of borel sets in $\mathbb{R}$.


You claim the following change of variables: $$\int_{X^{-1}(-\infty,a]} 1 d\mathbb{P} \overset{?}{~=~} \int_{\color{crimson}{X^{-1}(-\infty,a]}} \frac{d\mathbb{P}}{d \lambda} d\lambda$$

However, you should change the integration interval when you changed the variable.

$$\int_{X^{-1}(-\infty,a]} 1 d\mathbb{P} ~=~ \int_{\color{navy}{(-\infty,a]}} \frac{d\mathbb{P}}{d \lambda} d\lambda$$

That is all.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.