# What does the value of a probability density function (PDF) at some x indicate?

I understand that the probability mass function of a discrete random-variable X is $y=g(x)$. This means $P(X=x_0) = g(x_0)$.

Now, a probability density function of of a continuous random variable X is $y=f(x)$. Wikipedia defines this function $y$ to mean

In probability theory, a probability density function (pdf), or density of a continuous random variable, is a function that describes the relative likelihood for this random variable to take on a given value.

I am confused about the meaning of 'relative likelihood' because it certainly does not mean probability! The probability $P(X<x_0)$ is given by some integral of the pdf.

So what does $f(x_0)$ indicate? It gives a real number, but isn't the relative likelihood of a specific value for a CRV always zero?

• Let $f$ be the density function of $X$. Assume $f$ is continuous. Then if $h$ is small, the probability that $X$ lies in the interval $[a,a+h]$ is approximately $hf(a)$. By approximately I mean that the probability, divided by $h$, approaches $f(a)$ as $h$ approaches $0$. So the ratio $f(a)/f(b)$ measures, approximately, the ratio of the probability that $X$ is in $[a,a+h]$ to the probability $X$ is in $[b,b+h]$. – André Nicolas Oct 10 '12 at 19:28
• The value of $f$ is literally a probability density: the probability that $X$ lies in a small interval around $x_0$ is approximately $f(x_0)$ times the size of the interval. – user856 Jan 19 '19 at 6:10

'Relative likelihood' is indeed misleading. Look at it as a limit instead: $$f(x)=\lim_{h \to 0}\frac{F(x+h)-F(x)}{h}$$ where $F(x) = P(X \leq x)$

• So you suggest looking at the pdf as being defined by the cumulative distribution function? – jII Oct 10 '12 at 19:39
• This is essentially the definition of pdf fro CRVs – Alex Oct 10 '12 at 20:12
• A good way of thinking about is $f(x) = \frac{dF}{dx}$ and so it's the rate of change of the cdf at $x$. – Jacob Feb 27 '13 at 17:39
• Hi Alex, sorry for asking problem related to such an old answer. I know the pdf $f$ is the derivative of the cdf $F$, but what is the physical meaning of "the rate of change of the cdf at some point"? I mean, how to explain it by using the continuous random variable $X$? – Sam Wong Dec 13 '18 at 8:17
• I don't know much about physics sorry. – Alex Dec 13 '18 at 10:55

I am not sure if Jester is still interested, as it's been 5 years, but I think I found a less confusing anwer than in Wikipedia.

In contrast to discrete random variables, if X is continuous, f(X) is a function whose value at any given sample is not the probability but rather it indicates the likelihood that X will be in that sample/interval. For example if the value of the PDF around a point (can be generalized for a sample) x is large, that means the random variable X is more likely to take values close to x. If, on the other hand, f(x)=0 in some interval, then X won't be in that interval

Of course a more practical way of thinking it is that the probability of X being in an interval is given by the integral of the PDF.

You might want to look at the link below for more details: http://mathinsight.org/probability_density_function_idea

In general, if $X$ is a random variable with values of a measure space $(A,\mathcal A,\mu)$ and with pdf $f:A\to [0,1]$, then for all measurable set $S\in\mathcal A$, $$P(X\in S) = \int_S fd\mu$$ So, if $A=\Bbb R$ (and $\mu=\lambda$), then $$P(a<X<b)=\int_a^b f(x)dx$$ So, $f(x) = \displaystyle\lim_{t\to 0} \frac1{2t}\int_{x-t}^{x+t} f =\lim_{t\to 0} \frac1{2t} P(|X-x|<t)$ for example.. We can call it 'relative likelihood'..

• This is a better answer than Alex's but doesn't explain the significance of the number $f(x)$. Does it have a meaning independent of a cdf? Andre's answer of it being approximately $hf(a)$ is great but he doesn't indicate if there's more to $f(x)$ by itself. – Jacob Feb 27 '13 at 17:23

Intro statistics focuses on the PDF as the description of the population, but in fact it is the CDF (cumulative density function) that gives you a functional understanding of the population, as points on the CDF denote probabilities over a relevant range of measures. If you look at all stats from this perspective, then the PDF is just the description of probability change with respect to a change around a point along the measure at hand. The values on the PDF therefore only give you a look at the spread. For example, given two normal distributions $N(\mu_1, \sigma_1^2)$ and $N(\mu_2, \sigma_2^2)$, if you choose any value of $x$ to get point $p_n=\mu_n+x\cdot\sigma_n$ for the respective distributions and get $X_1[p_1 ] > X_2[p_2 ]$, then this just means $\sigma_1 < \sigma_2$. Similar relationships exist for other distributions.

For continuous probability distributions, it is not useful to talk about $$P(X=x_0)$$, since this probability is zero. It is only useful to talk about the probabilities of $$X$$ being in sets with a positive length, like a nontrivial interval, or a union of several intervals. So, instead of asking the probability of $$X=x_0$$, we ask the probability that $$X$$ is near $$x_0$$. To be precise, we talk about the probability that $$X$$ is in an interval of length $$\delta$$ centered at $$x_0$$, for some $$\delta>0$$. That is, $$P(x_0-\delta/2 As $$\delta\to 0$$, the above probability will approach zero. How quickly will it approach zero? It turns out that there will exist a number $$f(x_0)$$ for which $$P(x_0-\delta/2 This means that in $$P(x_0-\delta/2, the probability that $$X$$ in an interval around $$x_0$$ is approximated by $$f(x_0)$$ times the length of that interval. Furthermore, the relative error of this approximation approaches zero as the length of the interval approaches zero.

This qnuanty $$f(x_0)$$ is also the value of the pdf of $$X$$, and the preceding discussion is a way of giving it precise meaning.

The ratio of the pdf $f(x)$ at two points, $r_x = f(x_0)/f(x_1)$, is not a measure of relative probability (or "relative likelihood") for the two outcomes for the random variable $X$. The ratio depends on the metric. That is, with a variable transformation, $z=z(x)$, with the pdf for $Z$ given by $h(z)$, the ratio $r_z=h(z_0)/h(z_1)\neq r_x$, in general, even though the two ratios refer to the same two outcomes. For monotonic transformations, $f(x)\,dx = h(z)\,dz$.

Numerical values of the pdf have no value on their own. The metric, $dx$, is required for probability interpretations (ie. $f(x)\,dx$). Wikipedia got this wrong, so I have corrected it.