5
$\begingroup$

Consider a discrete case. PMF is the probability each value of random variable gets. So, for example, X ~ Poisson(2). I plot these probabilities (below), so I can say that I show the PMF of X. But on the other hand I show the distribution of X. For example, I can say whether the distribution I have is symmetrical or not. So, what is the difference between probability distribution and PMF terms (in discrete case)? Below I also bring the definitions from Wikipedia, but it is not helpful either.

Many thanks!

Enter image description here

A probability mass function (pmf) is a function that gives the probability that a discrete random variable is exactly equal to some value.

A probability distribution is a mathematical function that provides the probabilities of occurrence of different possible outcomes in an experiment.

$\endgroup$
3
$\begingroup$

I'm not aware of an agreed upon definition/meaning for probability distribution.

On the other hand, probability mass functions and probability density functions have agreed upon definitions and are used to describe probability distributions.

A probability density function is the generalization of probability mass functions to random variables which are not strictly discrete. In the case of a discrete random variable, the main difference is that the probability density function should integrate to one, while the probability mass function should add to one.

Suppose $X$ is a discrete random variable taking values $S=\{x_1,x_2,\ldots\} \subset \mathbb{R}$.

The probability mass function is a function $p : S\to [0,1]$ where $$ p(x) = \mathbb{P}(X=x) $$

On the other hand, the density function (of any RV) can be thought of as, $$ f(x)dx = \mathbb{P}(X\in[x+dx]) $$ In integral form you could write this as, $$ \int_{x}^{x+dx} f(z)dz = \mathbb{P}(X\in [x,x+dx]) $$

That is, the density times the width of a small interval gives the probability that $X$ is in that small interval $X\in[x,x+dx]$.

If the random variable is discrete, then the probability that $X$ is in this interval is the same as the probability $X=x$ for small enough $dx$. So you have $f(x)dx = \mathbb{P}(X=x)$ (or in integral form, $\lim_{dx\to 0}\int_{x}^{x+dx} f(z)dz = \mathbb{P}(X=x)$).

In particular, if $p(x)$ is the pmf for a discrete random variable $X$, then we can write the density function as: $$ f(x) = \sum_{i:p(x_i)\neq 0} p(x_i) \delta(x-x_i) $$ where $\delta(x)$ is the delta distribution; i.e. $\int_a^b f(x)\delta(c)d x = f(c)$ whenever $c\in[a,b]$

$\endgroup$
  • 2
    $\begingroup$ Only a continuous random variable has a density function. Note that $\lim_{dx \to 0} \int_x^{x+dx} f(z) dz = 0$, so your final equation states that $P(X = x) = 0$. This is true for a continuous random variable but not for a discrete random variable. (Discrete random variables don't have density functions.) $\endgroup$ – littleO Dec 31 '18 at 15:23
  • 2
    $\begingroup$ You can write the density of discrete random variables using delta distributions. $\endgroup$ – tch Dec 31 '18 at 15:26
2
$\begingroup$

The word "distribution" gets thrown around loosely sometimes, which can cause confusion.

The distribution of a random variable $X$ is the function that takes a set $S \subset \mathbb R$ as input and returns the number $P(X \in S)$ as output. (Technically I should assume that $S$ is a "nice" subset of $\mathbb R$ in some sense, but let's not worry about that.) I think the Wikipedia article would be more clear if it just gave us this definition up front.

The PMF of a random variable $X$ is the function that takes a number $x \in \mathbb R$ as input and returns the number $P(X=x)$ as output. If $X$ is a discrete random variable, then the PMF of $X$ is a convenient way to specify the distribution of $X$.

Here is one way to describe the relationship between the distribution of $X$ and the PMF of $X$, in the case where $X$ is a discrete random variable. Suppose that the possible values of $X$ are $x_1,x_2,\ldots$ If $f$ is the distribution of $X$, then $$ f(S) = \sum_{i : x_i \in S} P(X = x_i) $$ for any set $S \subset \mathbb R$.

$\endgroup$
  • 1
    $\begingroup$ Thanks. But is not the definition of PMF by itself exactly as you described for "distribution of a random variable X"? If not, how they are different? $\endgroup$ – John Dec 31 '18 at 14:34
  • 1
    $\begingroup$ No, those definitions are not the same. For example, the PMF of $X$ takes a number as input, but the distribution of $X$ takes a set of numbers as input. $\endgroup$ – littleO Dec 31 '18 at 14:38
  • 1
    $\begingroup$ Thank you a lot, for clarification. But I am still a bit confused regarding your statement "the distribution of X takes a set of numbers as input". Suppose I flip a coin twice, X is number of tails. My PMF is: 0: 1/4, 1: 1/2, 2, 1/4. So, for each value X (0,1,2,) the PMF returns the P(X=x). How will look the distribution function? It would help you can update your answer. Thanks! $\endgroup$ – John Dec 31 '18 at 14:51
  • 1
    $\begingroup$ Let $f$ be the distribution of the random variable $X$ that you mentioned. Then for example, if $S = (-7,1.3)$ (that is, $S$ is the open interval from $-7$ to $1.3$), then $f(S) = 3/4$. We could attempt to describe $f$ more explicitly but I'm not sure that it would be useful. The definition of $f$ is just that $f(S) = P(X \in S)$ for any set $S \subset \mathbb R$. $\endgroup$ – littleO Dec 31 '18 at 14:58
  • 1
    $\begingroup$ OK. I see. Thanks a lot. $\endgroup$ – John Dec 31 '18 at 15:03
0
$\begingroup$

I provide a simple explanation of this here: Difference between "probability density function" and "probability distribution function"?. In short, a probability mass function is a discrete probability distribution function, where discrete is often implied.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.