65
$\begingroup$

Wikipedia says:

The probability density function is nonnegative everywhere, and its integral over the entire space is equal to one.

and it also says.

Unlike a probability, a probability density function can take on values greater than one; for example, the uniform distribution on the interval $[0, \frac{1}{2}]$ has probability density $f(x) = 2$ for $0 ≤ x ≤ \frac{1}{2}$ and $f(x) = 0$ elsewhere.

How are these two things compatible?

$\endgroup$
5
  • 12
    $\begingroup$ It can take a value greater than $1$, but only over a region with measure less than $1$. $\endgroup$
    – Qiaochu Yuan
    Feb 3 '12 at 22:22
  • 4
    $\begingroup$ Compute the integral in your example. Is it 1? $\endgroup$
    – GEdgar
    Feb 3 '12 at 22:33
  • 10
    $\begingroup$ Think of it from a physics perspective: the probability density function is like the density of some compressible fluid, whereas it's integral is like the mass. If you take a fixed mass of liquid and compress certain parts of it, you can make the local density as high as you like without changing the total mass. $\endgroup$
    – Nick Alger
    Feb 4 '12 at 0:07
  • $\begingroup$ Yeah, I feel a bit silly now. $\endgroup$
    – zenna
    Feb 4 '12 at 1:05
  • $\begingroup$ @zenna I was confused by this too! $\endgroup$
    – Prince M
    Sep 12 '18 at 18:27
46
$\begingroup$

Consider the uniform distribution on the interval from $0$ to $1/2$. The value of the density is $2$ on that interval, and $0$ elsewhere. The area under the graph is the area of a rectangle. The length of the base is $1/2$, and the height is $2$ $$ \int\text{density} = \text{area of rectangle} = \text{base} \cdot\text{height} = \frac 12\cdot 2 = 1. $$

More generally, if the density has a large value over a small region, then the probability is comparable to the value times the size of the region. (I say "comparable to" rather than "equal to" because the value my not be the same at all points in the region.) The probability within the region must not exceed $1$. A large number---much larger than $1$---multiplied by a small number (the size of the region) can be less than $1$ if the latter number is small enough.

$\endgroup$
30
$\begingroup$

Remember that the 'PD' in PDF stands for "probability density", not probability. Density means probability per unit value of the random variable. That can easily exceed $1$. What has to be true is that the integral of this density function taken with respect to this value must be exactly $1$.

If we know a PDF function (e.g. normal distribution), and want to know the "probability" of a given value, say $x=1$, what will people usually do? To find the probability that the output of a random event is within some range, you integrate the PDF over this range.

Also see Why mvnpdf give probablity larger than 1?

$\endgroup$
1
  • 5
    $\begingroup$ Note that for a PDF, the probability that $x=1$ (or any other value) is $0$ because any point on the $x$-axis has $0$ length, so $\text{base} × \text{height} = 0$. For continuous probabilities it only makes sense to talk about probabilities near a point. For example, the probability that $x ∈ [0.9, 1.1]$ (i.e. $x=1±.1$) would be: $$∫_{0.9}^{1.1} f_X(x)\ dx$$ $\endgroup$
    – Zaz
    Apr 4 '16 at 23:31
14
$\begingroup$

I would like to point out an extreme example. Consider a probability density \begin{equation} f(x) = \begin{cases} \frac{1}{2d},& -d \leq x\leq +d \\ 0, & \text{elsewhere} \end{cases} \end{equation} Now, $\forall ~ d\neq0$
\begin{equation} \int_{-\infty}^{+\infty} f(x)dx = 1 \end{equation} Start with a smaller $d$ and find the integral again. It will be the same. It gets really interesting when we take the limit when $d \rightarrow0$ . The integral is again 1, but how does $f(x)$ looks like? \begin{equation} \lim_{d \to 0} f(x) = \begin{cases} \infty, & x=0 \\ 0, & \text{elsewhere} \end{cases} \end{equation} This limit is termed as the Dirac delta function $\delta(x)$, which is not really a function but the limit of a function. So the bottom line here

Not only the probability density can go greater than $1$, it can assume even bigger values (the biggest one is noted here) as long as the area under it is $1$.

$\endgroup$
2
$\begingroup$

Consider a probability density function of some continuous distribution. Why it is called density function??

Unlikely the probability mass function of a discrete distribution p.d.f doesn't have probability defined on a point. Conventionally probability of a point is considered as '0' for continuous distribution.

Then comes the question what actually does p.d.f give? P.d.f gives the probability of an interval, and integration over the support(domain of that variable it is defined on) of that variable is '1'.

Now the physical interpretation of density is ratio of mass and volume of a substance. Here probability density function is similarly the mass(which is similar to the frequency) and the volume(which is similar to the field of experiment) ratio.

So, it can be more than one and it can be anything.

For further discussion please ask me through gmail.

$\endgroup$
0
$\begingroup$

Continuous random variables can lead to confusion.

First, note that if $X$ is continuous then $P(X = x) = 0$ for every x. Don’t try to think of $f(x)$ as $P(X = x)$. This only holds for discrete random variables.

We get probabilities from a pdf by integrating.

A pdf can be bigger than $1$ (unlike a mass function).

For example, if $f(x) = 5 \text{ for $x \in [0, 1/5]$ and $0$ otherwise},$ then $f(x) \geq 0$ and $f(x)dx = 1$ so this is a well-defined pdf even though $f(x) = 5$ in some places.

In fact, a pdf can be unbounded.

For example, if $f(x) = (2/3)x−1/3$ for $0<x<1$ and $f(x) = 0$ otherwise, then $f(x)dx = 1,$ even though $f$ is not bounded.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.