1
$\begingroup$

This is a concept I have always struggled to understand:

  • When talking about Discrete Probability Functions, we usually have no problem assigning a probability to a single event. For example, when rolling a 6-sided die, we can say that the probability of encountering any of sides will have a certain probability.

  • However, when talking about Continuous Probability Functions, we are told that that idea of "individual probabilities" does not exist. For example, consider a random variable "X" that has a Normal Distribution with Mean = 0 and Sigma = 1. We can find out the probability of this random variable "X" taking values between 2.9 and 3.1 - but we can not find out the probability of this random variable "X" taking an exact value of 3.

I always wondered - what is the formal reason behind this?

For example:

  • We know that the "Mode" represents the most common number of a sample - and the "Mode" is a single number.

  • We could then find out the probability of a random variable taking values between "mode + 0.0001" and "mode - 0.0001" (e.g. via an integral) - and try to see if this number converges for smaller and smaller deviations from the "mode", thus finding out the probability of encountering exactly the mode.

I am aware that this logic that I have described is most likely flawed - but could someone please help me understand why the concept of "individual probabilities" does not exist in the continuous sense?

Thanks!

$\endgroup$
6
  • $\begingroup$ The probability of smaller and smaller intervals around the mode (or around any other particular value) will approach $0$. The expected value on those small intervals will approach the value the intervals are shrinking toward. The probability of any single value is $0$. That must be the case since the sum of the probabilities of all possible values must be $1$. That's why you need integration. $\endgroup$ Nov 2, 2022 at 1:19
  • 4
    $\begingroup$ Actually the probability that X takes a value of exactly 3 is well defined: it is zero. If that seems weird, you might be surprised to learn that the probability of X taking a value that is a rational number is also zero. If you want to understand this more deeply, perhaps the best place to start is learning the difference between countably and uncountably infinite sets. $\endgroup$
    – 3rdMoment
    Nov 2, 2022 at 1:27
  • $\begingroup$ The question you ask is closely related to measure theory. In a finite space, it's entirely possible for each singleton to have positive measure and still have the entire space have measure $1$. That's even possible in a countable space. But given a continuous probability measure, each singleton must have measure $0$ (because any point with positive measure would result in a jump discontinuity of the probability function). $\endgroup$ Nov 2, 2022 at 1:28
  • $\begingroup$ "... we are told that that idea of "individual probabilities" does not exist." Did someone really tell you this? Are you sure? Can you quote exactly what they said? $\endgroup$
    – David K
    Nov 2, 2022 at 1:32
  • 1
    $\begingroup$ We can indeed discuss the probability that $X = 3$. The probability is 0. $\endgroup$ Nov 2, 2022 at 1:42

1 Answer 1

2
$\begingroup$

However, when talking about Continuous Probability Functions, we are told that that idea of "individual probabilities" does not exist.

I don't know who told you this but it's not true. As mentioned in the comments, the individual probabilities exist, they're just equal to zero and so not very informative. In particular they cannot be used to define a sensible mode.

What does make sense is that if $X$ is a random variable with a continuous probability density function $f(x)$, $x$ is a particular value of $X$, and $\varepsilon > 0$ is arbitrary, we can compute the probability

$$\mathbb{P}(|X - x| < \varepsilon) = \int_{x - \varepsilon}^{x + \varepsilon} f(x) \, dx$$

that $X$ is within $\varepsilon$ of $x$. In the continuous case, as $\varepsilon$ becomes sufficiently small this becomes asymptotic to $2 \varepsilon f(x)$, and in fact the density can be defined using only probability measurements as

$$f(x) = \lim_{\varepsilon \to 0} \frac{\mathbb{P}(|X - x| < \varepsilon)}{2 \varepsilon}.$$

This definition should remind you of a derivative, and in fact it is called the Radon-Nikodym derivative. Then we can say that the mode, if it exists, is the value of $x$ that maximizes the probability density $f(x)$, which becomes the more useful and natural substitute for the probability in the continuous case. (However, note that, unlike in the discrete case, the notion of density and hence the notion of the mode is sensitive to reparameterization.)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .