Influencing Probabilities with Prior Information

Question

I have always had the following question about Bayesian Probabilities.

Suppose you observe the weather for 90 days. You observe that:

• 65 days it was Sunny
• 25 days it Rained

This means that there is a 0.72 probability of it being Sunny (65/90) and a 0.28 probability of it Raining (25/90) - to me, these probability formulas seem to be equivalent to the Maximum Likelihood Estimates.

My Question: Is there a way to "influence" these probabilities using a Bayesian Prior? Just like regression coefficients in a Bayesian Linear Regression model can be "influenced" using a Bayesian Prior - suppose we have reasons to believe that the true probabilities of these weather states have a Normal Distribution with mean = mu and variance = sigma - would it be possible to "factor" this information in and adjust our estimates of these weather frequencies?

For instance - could this perhaps result in P(Sunny) = 0.71 and P(Rain) = 0.29 ?

Thanks!

Note: I think a Binomial Distribution might be more suitable for a Prior Distribution in this example.

Answer 1

Looks like you are asking about Bayesian estimation. Bayes' rule tells us how to update a prior. You need a statistical model. For instance, suppose you use a Bernoulli model where $X_i$ is an indicator for a sunny day $i\in \{1,...,n\}$:

$$X_i|\theta \sim_{iid} Bern(\theta)$$

and $\theta$ is the unknown true probability that it is sunny. You also need a prior for $\theta\in [0,1].$ For instance, suppose your prior is a Beta prior (which is the conjugate prior for a Bernoulli likelihood):

$$\theta\sim Beta(\alpha,\beta)$$

for known hyperparameters $\alpha,\beta$.

Then by Bayes' rule, the posterior distribution is

$$\theta|X_1,...,X_n\sim Beta(\alpha+\sum_i X_i,\beta+n-\sum_i X_i).$$

You can work that out as an exercise or cheat and check the wiki table on conjugate priors.

The Bayes estimator for $\theta$ under quadratic loss would then be the mean of the posterior distribution. Since the mean of $Beta(A,B)$ is $A/(A+B)$ we have

$$\begin{align}E[\theta|X_1,...,X_n]&=\frac{\alpha+\sum_i X_i}{\alpha+\sum_i X_i+\beta+n-\sum_i X_i}\\&=\left(\frac{\alpha+\beta}{\alpha+\beta+n}\right)\frac{\alpha}{\alpha+\beta }+\left(\frac{n}{\alpha+\beta+n}\right)\frac{1}{n}\sum_i X_i\end{align}$$

which is a weighted average between the prior mean ($\alpha/(\alpha+\beta)$)and the ML estimator ($\frac{1}{n}\sum_i X_i$). This weighted averaging is common among Bayes estimators. Notice when $\alpha+\beta$ is large, the Bayes estimator leans more to the prior mean, but when your sample size is large it leans more toward the ML estimator.