I have always had the following question about Bayesian Probabilities.

Suppose you observe the weather for 90 days. You observe that:

  • 65 days it was Sunny
  • 25 days it Rained

This means that there is a 0.72 probability of it being Sunny (65/90) and a 0.28 probability of it Raining (25/90) - to me, these probability formulas seem to be equivalent to the Maximum Likelihood Estimates.

My Question: Is there a way to "influence" these probabilities using a Bayesian Prior? Just like regression coefficients in a Bayesian Linear Regression model can be "influenced" using a Bayesian Prior - suppose we have reasons to believe that the true probabilities of these weather states have a Normal Distribution with mean = mu and variance = sigma - would it be possible to "factor" this information in and adjust our estimates of these weather frequencies?

For instance - could this perhaps result in P(Sunny) = 0.71 and P(Rain) = 0.29 ?


Note: I think a Binomial Distribution might be more suitable for a Prior Distribution in this example.


1 Answer 1


Looks like you are asking about Bayesian estimation. Bayes' rule tells us how to update a prior. You need a statistical model. For instance, suppose you use a Bernoulli model where $X_i$ is an indicator for a sunny day $i\in \{1,...,n\}$:

$$X_i|\theta \sim_{iid} Bern(\theta)$$

and $\theta$ is the unknown true probability that it is sunny. You also need a prior for $\theta\in [0,1].$ For instance, suppose your prior is a Beta prior (which is the conjugate prior for a Bernoulli likelihood):

$$\theta\sim Beta(\alpha,\beta)$$

for known hyperparameters $\alpha,\beta$.

Then by Bayes' rule, the posterior distribution is

$$\theta|X_1,...,X_n\sim Beta(\alpha+\sum_i X_i,\beta+n-\sum_i X_i).$$

You can work that out as an exercise or cheat and check the wiki table on conjugate priors.

The Bayes estimator for $\theta$ under quadratic loss would then be the mean of the posterior distribution. Since the mean of $Beta(A,B)$ is $A/(A+B)$ we have

$$\begin{align}E[\theta|X_1,...,X_n]&=\frac{\alpha+\sum_i X_i}{\alpha+\sum_i X_i+\beta+n-\sum_i X_i}\\&=\left(\frac{\alpha+\beta}{\alpha+\beta+n}\right)\frac{\alpha}{\alpha+\beta }+\left(\frac{n}{\alpha+\beta+n}\right)\frac{1}{n}\sum_i X_i\end{align}$$

which is a weighted average between the prior mean ($\alpha/(\alpha+\beta)$)and the ML estimator ($\frac{1}{n}\sum_i X_i$). This weighted averaging is common among Bayes estimators. Notice when $\alpha+\beta$ is large, the Bayes estimator leans more to the prior mean, but when your sample size is large it leans more toward the ML estimator.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .