I am reading a book which says there are two ways of interpreting Bayes.

“In the Bayesian approach, parameters can be viewed from two perspectives. Either we view the parameters as truly varying, or we view our knowledge about the parameters as imperfect. The fact that we obtain different estimates of parameters from different studies can be taken to reflect either of these two views.”

“In the first case, we understand the parameters of interest as varying – taking on different values in each of the samples we pick . For example, suppose that we conduct a blood test on an individual in two consecutive weeks, and represent the correlation between the red and white cell count as a parameter of our statistical model. Due to the many factors that affect the body’s metabolism, the count of each cell type will vary somewhat randomly, and hence the parameter value may vary over time. In the second case, we view our uncertainty over a parameter’s value as the reason we estimate slightly different values in different samples. Bayesians are more at ease in using parameters as a means to an end – taking them not as real immutable constants, but as tools to help make inferences about a given situation.”

I dont get the second point, in the first due to the different conditions the parameter varies every time we measure. What does "we view our uncertainty over a parameter’s value as the reason we estimate slightly different values in different samples." mean? We measure different values each time, but what does this have to do with our certainty? Is he talking about our prediction probability? The yes we keep updating but that also reflects the distribution over the different conditions.

He then gives an example where frequentist makes sense but there is also a Bayes interpretation which I am not clear on:

“For example, if our parameter represented the probability that an individual taken at random from the UK population has dyslexia, it is reasonable to assume that there is a true, or fixed, population value of the parameter in question. While the Frequentist view may be reasonable here, the Bayesian view can also handle this situation. In Bayesian statistics these parameters can be assumed fixed, but that we are uncertain of their value (here the true prevalence of dyslexia) before we measure them, and use a probability distribution to reflect this uncertainty.”

Is he saying here that in a sample of 10, I would estimate 0% dyslexia, then in a 100, maybe it becomes 3% and in 1000 like 5%. I think the frequentist interpretation is apt here.

(Taken from Ben Lambert's 'A student's guide to Bayesian Statistics')

  • $\begingroup$ Could you please include a reference to the name of the book? It would provide some context. $\endgroup$ – Easymode44 Apr 17 '19 at 10:36

The main thing to consider is that the frequentist and Bayesian approaches give different answers depending on the specific question.

I will now interpret the dyslexia scenario in different ways, hopefully showing that there is no one approach but a family of approaches (which may or may not make sense to you depending on your application).

Interpretation 1 (Frequentist)

I have a population of $N$ people, of which $n$ have dyslexia. I need to estimate the ratio $n/N$ from a sample $M$ of the population. Note that, in this case, the true answer is $n/N$, and simply observing all the people (that is, taking $M=N$ without replacement), we have the true and unique and correct answer.

Interpretation 2 (Frequentist)

Each time a person is born, there is a probability $\alpha$ that this person is affected by dyslexia. The current population is a sample of the people-generating process and we can use it to estimate the probability $\alpha$. Note that, in this case, the true probability $\alpha$ is not measurable as even collecting all the people and measuring $n/N$ will not give exactly $\alpha$.

Interpretation 3 (Bayesian)

I believe that 1/10 people have some form of dyslexia. Given a sample of $M$ people (of which $m$ have dyslexia) from this population of $N$ people, how many people have dyslexia in the population? (Note that, also in this case, the Bayesian answer is $n$ if I have access to $M=N$ without replacement.)

Interpretation 4 (Bayesian)

I believe that the probability of a newborn having dyslexia is 0.1. Then, I observe $m$ people with dyslexia in a sample of $M$ people, what is my belief now?

  • $\begingroup$ Bayesians do not base their thoughts just on point estimates. So one might think his belief of the proportion of the population with dyslexia can be modelled with a Beta distribution with parameters $1$ and $9$ while another who (perhaps after updating earlier beliefs with observed evidence) might think her belief of the proportion of the population with dyslexia can be modelled with a Beta distribution with parameters $10$ and $90$. Both might give $0.1$ as a point estimate but the second has much less uncertainty about this value $\endgroup$ – Henry Apr 18 '19 at 8:05
  • $\begingroup$ This is very true. In my discourse I only considered point estimates; indeed also the uncertainty about the estimates will differ profoundly in the four approaches (and this is not only a Bayesian phenomenon). $\endgroup$ – Riccardo Sven Risuleo Apr 19 '19 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.