As an armchair academic, I am trying to teach myself some college level mathematics . Currently I am reading an introductory book into probability and statistics.

However, I am a bit confused by the following terminology, as (at least from my perspective) they seem to be used interchangeably.

Could someone help me clear up my misunderstandings as well as add anything I might have missed?

Vocabulary + my current understanding of them

1) probability mass function - function that returns the chance of a random variable taking on some value (for discrete only)

2) probability density function - function that returns the chance of a random variable taking on some value (for continuous only)

3) probability distribution - a general term that describes both #1 and #2

4) cumulative distribution function - a general term that describes both #1 and #2 aggregated over some range


Probability mass function is simply a function which returns a probability between $0$ and $1$ that events in the sample space will occur. For example, consider an unfair coin being flipped.

Say it has a $60\%$ chance of landing on heads, and $40\%$ chance of landing on tails. Lets call our sample space: $\Omega = \{H, T\}$. Where $H$ denotes a head, and $T$ denotes a tail.

Our probability mass function then is a function of the form $f:\Omega \rightarrow \mathbb{R}$, this can be written as:

$ f(x) = \begin{cases} 0.6 & x= H \\ \\ 0.4 & x = T \end{cases}$

Notice that $f(H) + f(T) = 1$

In fact, in general we must have: $$\sum_{x\in \Omega} f(x) = 1$$

This corresponds with the fact that something must happen.

As for probability density function, this is similar concept, except in this case we have a continuous sample space. In this case we have a function of the form $f:\Omega \rightarrow \mathbb{R}$, however in this case $\Omega$ is continuous. For our purposes we could take $\Omega$ to be $\mathbb{R}$.

In this case imagine the function $f$ is $f(x) = \frac{1}{\sqrt{2\pi}} e^{-x^2/2}$ for all $x \in \mathbb{R}$

Here we have an important distinction from the probability mass function. A particular output of the function $f(a)$ does NOT mean that $a$ has $f(a)$ chance of occurring. Rather it is a relative measurement, and gives its likelihood of occurring as compared to other values of $f(x)$. The probability of any particular value $a$ of occurring is in fact $0$. This may seem a bit strange but it is a result of measure theory and has to do with the size of infinity.

A probability distribution is just a blanket term for either of the first two mentioned.

The cumulative distribution function is essentially a function of the form $F: \Omega \rightarrow \mathbb{R}$ where for any $x \in \Omega$, $F(x)$ is the probability that an event less than or equal to $x$ will occur.

For example imagine we are rolling a fair six sided die. If $F$ is the cumulative distribution function, then $F(4)$ is the probability that anything less than or equal to a $4$ will be rolled. So if $f$ denotes the probability that a certain number will be rolled, then $F(4) = f(1) + f(2) + f(3) + f(4) = \frac{2}{3}$.

This is particularly useful when talking about probability density functions, the reason for which was alluded to earlier. In a probability density function the probability that any particular outcome of an experiment will occur is $0$. Imagine that the function we mentioned before $f(x) = \frac{1}{\sqrt{2\pi}} e^{-x^2/2}$ represents the probability that a random number generator will choose the number $x$.

In this case if we have a cumulative distribution function called $F$, then $F(x)$ denotes the probability that the number chosen is less than or equal to $x$. This is something useful that can give us a non-zero probability.

The function $F(x)$ would be written as: $$ F(x) = \int_{-\infty}^{x} f(t) dt$$

where $t$ is just a dummy variable of integration.

  • $\begingroup$ Presumably, you mean that $f(4)=2/3$ in your dice example. Normally capital letters are used for cumulative distribution functions and lower case letters for pdf's and pmf's. $\endgroup$ – Brian Borchers Feb 27 at 5:37
  • $\begingroup$ Yes, you are right I changed the calculation. I suppose I could change to a conventional upper case and lower case if that was desired. $\endgroup$ – Jack Pfaffinger Feb 27 at 5:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.