# the way of thinking of frequentist vs bayesian?

I'm learning about bayesian inference and I've heard that there's also another inference called frequentist inference. I still can't understand the difference the way frequentist and bayesian count the probability, can someone give me a simple example of a problem, and how frequentist and bayesian solve the problem?

Suppose you have polling data on Proposition A in an upcoming election. In a random sample of $n = 1000$ expected voters $x = 620$ people favor the proposition.

Frequentist analysis. There is some unknown fixed parameter $\theta,$ which is the probability a randomly chosen person favors the proposition. We have data from a binomial distribution with parameters $n = 1000$ and $\theta,$ and wish to estimate $\theta.$

The usual (maximum likelihood) point estimate is $\hat \theta = x/n = 620/1000 = 0.62.$ This estimate is based on maximizing the likelihood function $$p(x| \theta) = {n \choose x}\theta^x(1-\theta)^{n-x} \propto \theta^x(1-\theta)^{n-x}$$ with respect to $\theta.$ Of course, we do not expect $\theta$ to be exactly 0.62, so we seek an interval that expresses our uncertainty about this result.

Without going into details (available in almost all elementary statistics texts), a 95% confidence interval (CI), based on the normal approximation to binomial, is of the form $\hat \theta \pm 1.96\sqrt{\hat \theta(1-\hat \theta)/n}.$ Somewhat more precisely, we might use $\tilde n = n+4,$ and $\tilde \theta = (x+2)/\tilde n,$ to obtain the 'Agresti' confidence interval $\tilde \theta \pm 1.96\sqrt{\tilde \theta(1-\tilde \theta)/\tilde n}.$ The Agresi confidence interval for our poll is $(0.5895, 0.6496).$

Roughly, the idea is that there are $n+1$ possible CIs, one for each of the possible values of $x.$ Some of them 'cover' (include) the 'true' value of $\theta,$ and some don't. For given values of $n$ and $\theta,$ the binomial probability ${n \choose x}\theta^x(1-\theta)^{n-x}$ is associated with each CI. Ideally (and for the Agresi-style interval nearly actually), the intervals that cover the true value of $\theta$ have total probability 95%.

Thus the process by which the CI is constructed implies that it covers the true value of $\theta$ with probability 95%. To a frequentist, this means that the process 'works' 95% of the time, over the long run.

Therefore, once $x$ is observed and a particular CI is constructed, one must not say that the true $\theta$ lies in the that CI with 95% probability. Either it lies in the interval or it does not. But we are allowed to say we have 95% 'confidence' that the CI covers the true $\theta.$

Bayesian analysis. You say you are familiar with Bayesian ideas, so I will be briefer here. The crucial distinction is that we begin by regarding $\theta$ as a random variable (not an unknown fixed constant). If you have no useful prior information about Proposition A or the attitudes of the electorate, you might select an uninformative prior such as $Unif(0,1) \equiv Beta(\alpha_0=1, \beta_0 =1).$

The likelihood function is as in the frequentist discussion above. So the posterior distribution is given by $$p(\theta|x) = p(\theta)\times p(x|\theta) \propto \theta^{\alpha_0 -1}(1-\theta)^{\beta_0-1} \times \theta^x(1-\theta)^{n-x} \propto \theta^{621-1}(1-\theta)^{381-1},$$

which we recognize as the kernel of $Beta(621, 381)$. We might use the mean, median, or mode of the posterior distribution as a point estimate for $\theta$. A probability symmetric Baysian probability interval for $\theta$ is $(0.5895, 0.6496).$

qbeta(c(.025,.975), 621, 381)
## 0.5894984 0.6495697


However, because an initial probability distribution for the random variable $\theta$ is provided by the prior, we may say there is probability 95% that $\theta$ lies in the stated interval in the particular instance at hand.

Of course, if you have some meaningful prior information, you might choose an informative prior. perhaps the mildly optimistic $Beta(330, 270)$. Then the 'Bayesian probability interval' would be $(0.5696, 0.6177).$ Sometimes such intervals are called 'Bayesian credible intervals'.

Acknowledgment: The Bayesian example is condensed from Suess & Trumbo (2010), Ch 8. Coverage probabilities for frequentist binomial CIs are discussed in Ch 1.