My definition of empirical probability:

Let $(X_1,\ldots,X_n)$ be a random sample of i.i.d random variables. Then, the empirical probability for a subset A contained in the sample space $\Omega$ is $P_n(A)=\frac{1}{n}\sum_{i=1}^n \mathbb{1}(X_i \in A).$ Fine.

What I don't get is that $P_n(A)$ is a random variable for a fixed event A. Therefore, won't it have it's own (theoretical) probability distribution?

My understanding of empirical probability with an example:

Say you roll a 6 sided die and you introduce the random variable X, where X=1 if you roll a $1$, $X=2$ if you roll a $2$ etc.

You don't know anything about how $X$ is distributed (it may or may not be a fair die for example).

Roll the same die $n$ times, producing the random sample $(X_1,\ldots,X_n)$

Let $A=\{1\}$ and $P(X \in A)=p$. By defintion, $P_n(A) = \{\text{counting the number of times 1 appears}\}/n$. Since the indicator function in this case is a random variable that has a Bernoulli distribution with parameter $p$, $nP_n(A)$ is just a random variable that has a binomial distribution with parameters $p$ and $n$.

Why then is the empirical probability called "empirical" as if to suggest its distribution would be generated by experiment? I kind of just view this as related to (if not a generalization of) the sampling distribution of sample proportions, where $X_i$ are not necessarily Bernoulli.


1 Answer 1


$P_n(A)$ is a random variable and it does have its own probability distribution, if you know (or impose) a distribution on the sample. That is precisely what you did when you set $\Pr[X \in A] = p$. But the reason why $P_n(A)$ is empirical is because, a priori, there is no assumption on the sampling distribution: the calculation of the statistic $P_n(A)$ does not depend on any parametric assumptions. You don't necessarily know or assume anything about the model from which the observations in your sample were generated.

  • $\begingroup$ Thanks for the response. I see what you're saying, but I haven't defined what p is (I've only assumed that it's non-zero). Can we still not make the claim that $P_n(A)$ is binomially distributed, just centered in different places? $\endgroup$
    – Winston
    Jul 21, 2016 at 19:15
  • $\begingroup$ @Winston Unfortunately, no. You're implicitly assuming that the random variable $$\mathbb 1(X_i \in A) \sim \operatorname{Bernoulli}(p),$$ and furthermore, that these are IID. Although a wide variety of models could be chosen for $X_i$ itself, the point is that $p$ here is a parameter, thus a distributional assumption is being made. $\endgroup$
    – heropup
    Jul 21, 2016 at 19:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .