23
$\begingroup$

I understand that the probability mass function of a discrete random-variable X is $y=g(x)$. This means $P(X=x_0) = g(x_0)$.

Now, a probability density function of of a continuous random variable X is $y=f(x)$. Wikipedia defines this function $y$ to mean

In probability theory, a probability density function (pdf), or density of a continuous random variable, is a function that describes the relative likelihood for this random variable to take on a given value.

I am confused about the meaning of 'relative likelihood' because it certainly does not mean probability! The probability $P(X<x_0)$ is given by some integral of the pdf.

So what does $f(x_0)$ indicate? It gives a real number, but isn't the relative likelihood of a specific value for a CRV always zero?

$\endgroup$
2
  • 5
    $\begingroup$ Let $f$ be the density function of $X$. Assume $f$ is continuous. Then if $h$ is small, the probability that $X$ lies in the interval $[a,a+h]$ is approximately $hf(a)$. By approximately I mean that the probability, divided by $h$, approaches $f(a)$ as $h$ approaches $0$. So the ratio $f(a)/f(b)$ measures, approximately, the ratio of the probability that $X$ is in $[a,a+h]$ to the probability $X$ is in $[b,b+h]$. $\endgroup$ Commented Oct 10, 2012 at 19:28
  • $\begingroup$ The value of $f$ is literally a probability density: the probability that $X$ lies in a small interval around $x_0$ is approximately $f(x_0)$ times the size of the interval. $\endgroup$
    – user856
    Commented Jan 19, 2019 at 6:10

6 Answers 6

16
$\begingroup$

'Relative likelihood' is indeed misleading. Look at it as a limit instead: $$ f(x)=\lim_{h \to 0}\frac{F(x+h)-F(x)}{h} $$ where $F(x) = P(X \leq x)$

$\endgroup$
5
  • $\begingroup$ So you suggest looking at the pdf as being defined by the cumulative distribution function? $\endgroup$
    – jII
    Commented Oct 10, 2012 at 19:39
  • 2
    $\begingroup$ This is essentially the definition of pdf fro CRVs $\endgroup$
    – Alex
    Commented Oct 10, 2012 at 20:12
  • 3
    $\begingroup$ A good way of thinking about is $f(x) = \frac{dF}{dx}$ and so it's the rate of change of the cdf at $x$. $\endgroup$
    – Jacob
    Commented Feb 27, 2013 at 17:39
  • $\begingroup$ Hi Alex, sorry for asking problem related to such an old answer. I know the pdf $f$ is the derivative of the cdf $F$, but what is the physical meaning of "the rate of change of the cdf at some point"? I mean, how to explain it by using the continuous random variable $X$? $\endgroup$
    – Sam Wong
    Commented Dec 13, 2018 at 8:17
  • $\begingroup$ I don't know much about physics sorry. $\endgroup$
    – Alex
    Commented Dec 13, 2018 at 10:55
8
$\begingroup$

I am not sure if Jester is still interested, as it's been 5 years, but I think I found a less confusing anwer than in Wikipedia.

In contrast to discrete random variables, if X is continuous, f(X) is a function whose value at any given sample is not the probability but rather it indicates the likelihood that X will be in that sample/interval. For example if the value of the PDF around a point (can be generalized for a sample) x is large, that means the random variable X is more likely to take values close to x. If, on the other hand, f(x)=0 in some interval, then X won't be in that interval

Of course a more practical way of thinking it is that the probability of X being in an interval is given by the integral of the PDF.

You might want to look at the link below for more details: http://mathinsight.org/probability_density_function_idea

$\endgroup$
4
$\begingroup$

In general, if $X$ is a random variable with values of a measure space $(A,\mathcal A,\mu)$ and with pdf $f:A\to [0,1]$, then for all measurable set $S\in\mathcal A$, $$P(X\in S) = \int_S fd\mu $$ So, if $A=\Bbb R$ (and $\mu=\lambda$), then $$P(a<X<b)=\int_a^b f(x)dx$$ So, $f(x) = \displaystyle\lim_{t\to 0} \frac1{2t}\int_{x-t}^{x+t} f =\lim_{t\to 0} \frac1{2t} P(|X-x|<t) $ for example.. We can call it 'relative likelihood'..

$\endgroup$
1
  • 1
    $\begingroup$ This is a better answer than Alex's but doesn't explain the significance of the number $f(x)$. Does it have a meaning independent of a cdf? Andre's answer of it being approximately $hf(a)$ is great but he doesn't indicate if there's more to $f(x)$ by itself. $\endgroup$
    – Jacob
    Commented Feb 27, 2013 at 17:23
3
$\begingroup$

Intro statistics focuses on the PDF as the description of the population, but in fact it is the CDF (cumulative density function) that gives you a functional understanding of the population, as points on the CDF denote probabilities over a relevant range of measures. If you look at all stats from this perspective, then the PDF is just the description of probability change with respect to a change around a point along the measure at hand. The values on the PDF therefore only give you a look at the spread. For example, given two normal distributions $N(\mu_1, \sigma_1^2)$ and $N(\mu_2, \sigma_2^2)$, if you choose any value of $x$ to get point $p_n=\mu_n+x\cdot\sigma_n$ for the respective distributions and get $X_1[p_1 ] > X_2[p_2 ]$, then this just means $\sigma_1 < \sigma_2$. Similar relationships exist for other distributions.

$\endgroup$
1
2
$\begingroup$

For continuous probability distributions, it is not useful to talk about $P(X=x_0)$, since this probability is zero. It is only useful to talk about the probabilities of $X$ being in sets with a positive length, like a nontrivial interval, or a union of several intervals. So, instead of asking the probability of $X=x_0$, we ask the probability that $X$ is near $x_0$. To be precise, we talk about the probability that $X$ is in an interval of length $\delta$ centered at $x_0$, for some $\delta>0$. That is, $$ P(x_0-\delta/2<X<x_0+\delta/2) $$ As $\delta\to 0$, the above probability will approach zero. How quickly will it approach zero? It turns out that there will exist a number $f(x_0)$ for which $$ P(x_0-\delta/2<X<x_0+\delta/2)\approx \delta f(x_0) $$ This means that in $P(x_0-\delta/2<X<x_0+\delta/2)$, the probability that $X$ in an interval around $x_0$ is approximated by $f(x_0)$ times the length of that interval. Furthermore, the relative error of this approximation approaches zero as the length of the interval approaches zero.

This qnuanty $f(x_0)$ is also the value of the pdf of $X$, and the preceding discussion is a way of giving it precise meaning.

$\endgroup$
0
$\begingroup$

The ratio of the pdf $f(x)$ at two points, $r_x = f(x_0)/f(x_1)$, is not a measure of relative probability (or "relative likelihood") for the two outcomes for the random variable $X$. The ratio depends on the metric. That is, with a variable transformation, $z=z(x)$, with the pdf for $Z$ given by $h(z)$, the ratio $r_z=h(z_0)/h(z_1)\neq r_x$, in general, even though the two ratios refer to the same two outcomes. For monotonic transformations, $f(x)\,dx = h(z)\,dz$.

Numerical values of the pdf have no value on their own. The metric, $dx$, is required for probability interpretations (ie. $f(x)\,dx$). Wikipedia got this wrong, so I have corrected it.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .