16
$\begingroup$

Perhaps this is but a subtlety but I've noticed that in quite a few definitions in statistics and probability, definitions regarding the distribution of a variable or a sample of data choose to use the cumulative distribution function to characterise random variables as following a specific distribution as opposed to using the probability density function.

E.g with the Kolmogorov Smirnov test, we look at the difference between cdfs and not pdfs.

Is there a specific reason for this ?

$\endgroup$
  • 7
    $\begingroup$ because densities doesn't exists in general, however the distribution of a random variable ever exists $\endgroup$ – Masacroso Dec 12 '19 at 15:20
  • $\begingroup$ Singular distributions. $\endgroup$ – copper.hat Dec 12 '19 at 23:27
  • $\begingroup$ As a practical matter, it's a lot easier to transform a variable using its cumulative distribution than it is to use its probability density. There are just fewer things you have to remember correctly. That's why I personally prefer them. I suspect there are deeper reasons if you're actually going into the theory, though. $\endgroup$ – Arthur Dec 13 '19 at 10:20
33
$\begingroup$

Every random variable has a CDF. Not every random variable has a pdf (for instance, discrete or mixtures of discrete and continuous distributions). For instance, in the Kolmogorov-Smirnov test, you are comparing an empirical CDF, which is discrete, to a potentially continuous CDF. In addition, convergence in distribution is defined using CDFs.

$\endgroup$
  • 3
    $\begingroup$ I would however argue that a) every random variable does have a probability density, just it's not always a function, b) while it is always possible to give a CDF, .this requires the domain to be ordered, for which there is in general no straightforward way (in particular for multidimensional continuous distributions). $\endgroup$ – leftaroundabout Dec 13 '19 at 8:48
  • 2
    $\begingroup$ Multivariate CDFs exist. $\endgroup$ – kccu Dec 13 '19 at 13:17
  • 2
    $\begingroup$ So does a multivariate pdf, so that's not really an argument against CDFs. $\endgroup$ – kccu Dec 13 '19 at 13:23
  • 7
    $\begingroup$ @drhab it's a distribution. This can be formalised perfectly well in the framework of differential forms, and in such a way that allows describing discrete random variables on a continuous space (cf. Dirac distribution), and ... $\endgroup$ – leftaroundabout Dec 13 '19 at 14:33
  • 4
    $\begingroup$ @leftaroundabout This question is about statistics, not physics. $\endgroup$ – kccu Dec 13 '19 at 18:21
14
$\begingroup$

The CDF of a random variable completely determines the distribution of a random variable.

The PDF of a random variable completely determines the distribution of a random variable.


So uptil here no differences, but they arise if we look at the converse of these statements.

The distribution of a random variable is determining for its CDF, or - in other words - there is only one CDF.

This cannot be said about PDF.

Observe for instance that for uniform distribution on $[0,1]$ we can use several PDF's. The most common one is the indicator function of $[0,1]$ but another is the indicator function of $[0,1]\cap\mathbb Q^{\complement}$.


The CDF of a random variable always exists.

The PDF of a random variable does not always exist.

$\endgroup$
  • 8
    $\begingroup$ This doesn't really make sense. The PDF is unique outside of a Lebesgue null set. You want to argue that the CDF is unique everywhere, but it isn't for the cases where the distinction is actually important, namely discrete distributions. The CDF of, say, the outcomes of a dice throw seen in $\mathbb{R}$ is defined uniquely almost everywhere, but at e.g. 2 it is undefined within the range $[\frac16,\frac13]$, at 4 undefined within $[\tfrac23,\tfrac56]$ etc.. $\endgroup$ – leftaroundabout Dec 13 '19 at 9:08
  • 3
    $\begingroup$ @lelftaroundabout For every real-valued random variable $X$ and every $x\in\mathbb R$ the event $\{X\leq x\}$ is measurable so that $F_X(x)=P(X\leq x)$ exists, is (of course) unique and is properly defined. What you are saying about the CDF of a random variable that represents the outcome of the roll of a die makes no sense at all. If the die is fair then in that case $F_X(2)=P(X=1)+P(X=2)=\frac26$. $\endgroup$ – drhab Dec 13 '19 at 14:13
  • 1
    $\begingroup$ @leftaroundabout Further on your first point if $f,g$ are both Pdf's of the same distribution then the set $\{f\neq g\}$ is a Borel set with Lebesgue measure $0$. I do not deny that in my answer but just remark that we do not necessarily have $f=g$. Why do you label that as something that doesn't really make sense?... $\endgroup$ – drhab Dec 13 '19 at 14:15
  • 1
    $\begingroup$ @leftaroundabout For completeness: CDF as used in my answer is defined here. $\endgroup$ – drhab Dec 13 '19 at 14:30
  • 1
    $\begingroup$ Don't really understand the confusion; this is pretty much a complete answer. $\endgroup$ – StubbornAtom Dec 14 '19 at 14:19
13
$\begingroup$

You pretty much always want to calculate Pr(a < x < b), which is a simple subtraction if you have a CDF. If you're using a PDF, you have to integrate, which is a very complex operation.

$\endgroup$
  • 3
    $\begingroup$ Simple as that, this is IMO the best argument. Though of course the same caveat as for the other answers applies: this only works for 1D distributions. For continuous distributions on a higher-dimensional domain, CDF is not straightforward at all, neither to define nor to use. $\endgroup$ – leftaroundabout Dec 13 '19 at 12:55
  • 1
    $\begingroup$ @leftaroundabout Also for random vectors we have CDF's. Maybe less easy to use but defined on a straightforward and proper way. $\endgroup$ – drhab Dec 13 '19 at 14:25
  • $\begingroup$ Maybe we should note that $P(a < X \leq b) = F(b) - F(a)$ (where $F$ is the CDF of $X$). If $X$ is not continuous, then using $F$ to compute $P(a < X < b)$ is not quite so simple. $\endgroup$ – littleO Dec 13 '19 at 18:55
5
$\begingroup$

The values of a CDF are probabilities, and thus easy to directly reason about. On the other hand the values of a PDF (when it exists) have no direct intuitive meaning but are instead characterized less directly by the property that when you integrate over a set you get the probability of being in that set. That is a little harder to reason about. CDFs are thus more intuitive than PDFs.

On the other hand, in the discrete case both CDFs and PMFs can be directly interpreted as probabilities, with the PMF giving the probability of a simpler event. In such situations the PMF is often both easier to work with and perhaps more intuitive.

$\endgroup$
  • 2
    $\begingroup$ The values of the PDF can be thought of as "probability per X", where X is the unit of the domain. $\endgroup$ – Jack M Dec 14 '19 at 9:33
-2
$\begingroup$

A CDF behaves straightforwardly under transformation/substitution of variables while a PDF requires meddling with differential coefficients.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.