I've read multiple of the threads about Bayesian vs Frequentist interpretations of probability, but I'm having trouble trying to reconcile them with the idea of the sampling distribution when performing parameter estimation. Here is what is going on in my head:

Say we want to fit a probability distribution to a collection of data. Examining the data we choose a probability distribution (say the Poisson as an example). To fit the distribution to our data we have to obtain an estimate for the population parameter $\lambda$. So we obtain the estimate using one of the known procedures (e.g. Maximum Liklihood Estimation). We obtain an estimate for the true parameter value, $\hat{\lambda}$. Since our estimate $\hat{\lambda}$ is a function of our sampled data this implies it has a probability distribution, which is commonly referred to as the sampling distribution.

Now this is where I'm having trouble tying things together. My interpretation of what a sampling distribution was that it "represents the possible values that the true parameter value could take on with respective probability attached to them." But also from the frequentist approach the parameter values are constant and as such have no probability attached to them.

But we obtained these parameter estimates from a random process of sampling, so how can they not have probability attached to them?

As can be seen there a few things I'm confused about. Some clarification would be much appreciated.

  • $\begingroup$ In the frequentist approach, the parameter $\lambda$ is just a number (a fixed constant, not random), but our estimator $\hat \lambda$ is a random variable. I would throw out your interpretation of the sampling distribution and replace it with this: the sampling distribution for $\hat \lambda$ just tells us how likely $\hat \lambda$ is to take on various values. $\endgroup$
    – littleO
    Mar 13, 2020 at 1:43

1 Answer 1


In the frequentist interpretation, parameters are fixed but unknown, so it is correct to say that they are not stochastic. We infer their values from realizations of the distribution they model, and this is what we call parameter estimation. The fundamental question is, "where is the randomness?"

A frequentist answers this question by saying the parameters are fixed but the observations/data are realizations of some underlying random process. The data are random in the sense that they change from experiment to experiment by virtue of some random sampling process. Therefore, any statistic we calculate from the data inherits that randomness, including any statistics that are parameter estimates. This includes point estimates, interval estimates, and so forth.

So the sampling distribution of an estimator, or of any statistic for that matter, is the probability distribution of that statistic. It is not, as you have put it, a representation of "the possible values that the true parameter value could take on," in as much as it is incorrect to speak of a confidence interval's coverage probability as the probability that the true value of the parameter falls within a given calculated interval.

Sampling distributions are distributions on statistics, and in the case where a statistic is an estimator of a parameter, they are distributions on those estimators. They are not distributions of parameters.

To illustrate, it is a rather unfortunate and common abuse of language to use the phrase "sampling distribution of the mean" when in fact the terminology ought to be "sampling distribution of the sample mean." The former is misleading because it suggests that a distribution exists for the parameter(s) or some function thereof (the mean of the distribution), rather than for a statistic obtained from data, which is the sample mean. It seems redundant to use "sampling"/"sample" but I argue that the redundancy here is "sampling," not "sample," since the meaning of "distribution of the sample mean" cannot be misunderstood as anything else than a sampling distribution. Similarly, we can construct sampling distributions for the standard error (of the mean), or for the sample median, and it is not at all ambiguous to say "distribution of the sample median" without prefacing it with "sampling."

When we use a sampling distribution for constructing an interval estimate, e.g., the standard error of the sampling distribution of the sample mean may give us a confidence interval for a parameter, the resulting estimate is itself a random variable (or pair of random variables, if you want to think of it that way). The coverage probability is the probability that the interval contains the true value of the parameter. The distinction is subtle but important: here, the parameter is fixed, and it is the interval that varies randomly from one experiment/sample to another, and the coverage probability is what proportion of such samples on average will produce an interval estimate which contains the parameter being estimated.

  • $\begingroup$ I don't know if it's because I asked the question compared to just reading other responses elsewhere, but this explanation has brought me ALOT of clarity. One quick follow up: Since we have the sampling distribution of the statistic (estimator) and our final objective is to use the best estimator possible, wouldn't we want to use the estimate that has the highest probability of occurring?.....I guess this would be optimizing our sampling distribution...and if i am looking at it correctly that is exactly what getting a MLE is doing.. $\endgroup$ Mar 13, 2020 at 2:30
  • $\begingroup$ @dc3rd The "best" estimator is something that is not necessarily easy to quantify in an objective sense. Depending on the properties of an estimator (i.e., the shape of its sampling distribution!), it may be preferable to choose one that is biased but has smaller variance over an unbiased estimator with larger variance. $\endgroup$
    – heropup
    Mar 13, 2020 at 2:36
  • $\begingroup$ I see. That makes sense to me. Thanks for your assistance. $\endgroup$ Mar 13, 2020 at 2:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .