22
$\begingroup$

Perhaps this is but a subtlety but I've noticed that in quite a few definitions in statistics and probability, definitions regarding the distribution of a variable or a sample of data choose to use the cumulative distribution function to characterise random variables as following a specific distribution as opposed to using the probability density function.

E.g with the Kolmogorov Smirnov test, we look at the difference between cdfs and not pdfs.

Is there a specific reason for this ?

$\endgroup$
3
  • 9
    $\begingroup$ because densities doesn't exists in general, however the distribution of a random variable ever exists $\endgroup$
    – Masacroso
    Dec 12, 2019 at 15:20
  • $\begingroup$ Singular distributions. $\endgroup$
    – copper.hat
    Dec 12, 2019 at 23:27
  • 1
    $\begingroup$ As a practical matter, it's a lot easier to transform a variable using its cumulative distribution than it is to use its probability density. There are just fewer things you have to remember correctly. That's why I personally prefer them. I suspect there are deeper reasons if you're actually going into the theory, though. $\endgroup$
    – Arthur
    Dec 13, 2019 at 10:20

5 Answers 5

Reset to default
36
$\begingroup$

Every random variable has a CDF. Not every random variable has a pdf (for instance, discrete or mixtures of discrete and continuous distributions). For instance, in the Kolmogorov-Smirnov test, you are comparing an empirical CDF, which is discrete, to a potentially continuous CDF. In addition, convergence in distribution is defined using CDFs.

$\endgroup$
13
  • 3
    $\begingroup$ I would however argue that a) every random variable does have a probability density, just it's not always a function, b) while it is always possible to give a CDF, .this requires the domain to be ordered, for which there is in general no straightforward way (in particular for multidimensional continuous distributions). $\endgroup$
    – leftaroundabout
    Dec 13, 2019 at 8:48
  • 2
    $\begingroup$ Multivariate CDFs exist. $\endgroup$
    – kccu
    Dec 13, 2019 at 13:17
  • 2
    $\begingroup$ So does a multivariate pdf, so that's not really an argument against CDFs. $\endgroup$
    – kccu
    Dec 13, 2019 at 13:23
  • 7
    $\begingroup$ @drhab it's a distribution. This can be formalised perfectly well in the framework of differential forms, and in such a way that allows describing discrete random variables on a continuous space (cf. Dirac distribution), and ... $\endgroup$
    – leftaroundabout
    Dec 13, 2019 at 14:33
  • 4
    $\begingroup$ @leftaroundabout This question is about statistics, not physics. $\endgroup$
    – kccu
    Dec 13, 2019 at 18:21
20
$\begingroup$

You pretty much always want to calculate Pr(a < x < b), which is a simple subtraction if you have a CDF. If you're using a PDF, you have to integrate, which is a very complex operation.

$\endgroup$
3
  • 4
    $\begingroup$ Simple as that, this is IMO the best argument. Though of course the same caveat as for the other answers applies: this only works for 1D distributions. For continuous distributions on a higher-dimensional domain, CDF is not straightforward at all, neither to define nor to use. $\endgroup$
    – leftaroundabout
    Dec 13, 2019 at 12:55
  • 2
    $\begingroup$ @leftaroundabout Also for random vectors we have CDF's. Maybe less easy to use but defined on a straightforward and proper way. $\endgroup$
    – drhab
    Dec 13, 2019 at 14:25
  • 2
    $\begingroup$ Maybe we should note that $P(a < X \leq b) = F(b) - F(a)$ (where $F$ is the CDF of $X$). If $X$ is not continuous, then using $F$ to compute $P(a < X < b)$ is not quite so simple. $\endgroup$
    – littleO
    Dec 13, 2019 at 18:55
17
$\begingroup$

The CDF of a random variable completely determines the distribution of a random variable.

The PDF of a random variable completely determines the distribution of a random variable.

So uptil here no differences, but they arise if we look at the converse of these statements.

The distribution of a random variable is determining for its CDF, or - in other words - there is only one CDF.

This cannot be said about PDF.

Observe for instance that for uniform distribution on $[0,1]$ we can use several PDF's. The most common one is the indicator function of $[0,1]$ but another is the indicator function of $[0,1]\cap\mathbb Q^{\complement}$.

The CDF of a random variable always exists.

The PDF of a random variable does not always exist.

$\endgroup$
5
  • 9
    $\begingroup$ This doesn't really make sense. The PDF is unique outside of a Lebesgue null set. You want to argue that the CDF is unique everywhere, but it isn't for the cases where the distinction is actually important, namely discrete distributions. The CDF of, say, the outcomes of a dice throw seen in $\mathbb{R}$ is defined uniquely almost everywhere, but at e.g. 2 it is undefined within the range $[\frac16,\frac13]$, at 4 undefined within $[\tfrac23,\tfrac56]$ etc.. $\endgroup$
    – leftaroundabout
    Dec 13, 2019 at 9:08
  • 4
    $\begingroup$ @lelftaroundabout For every real-valued random variable $X$ and every $x\in\mathbb R$ the event $\{X\leq x\}$ is measurable so that $F_X(x)=P(X\leq x)$ exists, is (of course) unique and is properly defined. What you are saying about the CDF of a random variable that represents the outcome of the roll of a die makes no sense at all. If the die is fair then in that case $F_X(2)=P(X=1)+P(X=2)=\frac26$. $\endgroup$
    – drhab
    Dec 13, 2019 at 14:13
  • 2
    $\begingroup$ @leftaroundabout Further on your first point if $f,g$ are both Pdf's of the same distribution then the set $\{f\neq g\}$ is a Borel set with Lebesgue measure $0$. I do not deny that in my answer but just remark that we do not necessarily have $f=g$. Why do you label that as something that doesn't really make sense?... $\endgroup$
    – drhab
    Dec 13, 2019 at 14:15
  • 2
    $\begingroup$ @leftaroundabout For completeness: CDF as used in my answer is defined here. $\endgroup$
    – drhab
    Dec 13, 2019 at 14:30
  • 2
    $\begingroup$ Don't really understand the confusion; this is pretty much a complete answer. $\endgroup$
    – StubbornAtom
    Dec 14, 2019 at 14:19
6
$\begingroup$

The values of a CDF are probabilities, and thus easy to directly reason about. On the other hand the values of a PDF (when it exists) have no direct intuitive meaning but are instead characterized less directly by the property that when you integrate over a set you get the probability of being in that set. That is a little harder to reason about. CDFs are thus more intuitive than PDFs.

On the other hand, in the discrete case both CDFs and PMFs can be directly interpreted as probabilities, with the PMF giving the probability of a simpler event. In such situations the PMF is often both easier to work with and perhaps more intuitive.

$\endgroup$
1
  • 2
    $\begingroup$ The values of the PDF can be thought of as "probability per X", where X is the unit of the domain. $\endgroup$
    – Jack M
    Dec 14, 2019 at 9:33
-1
$\begingroup$

A CDF behaves straightforwardly under transformation/substitution of variables while a PDF requires meddling with differential coefficients.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .