The approach and interpretation of probability associated with Bayes theorem; usually used as opposed to the frequentist approach. It can be seen as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain. A Bayesian probabilist starts with some prior ...

learn more… | top users | synonyms

12
votes
6answers
5k views

Bayes Theorem Example in Nate Silver's The Signal and the Noise

In his book The Signal and the Noise, Nate Silver presents this example application of Bayes's Theorem on pp. 247-248: Consider a somber example: the September 11 attacks. Most of us would have ...
9
votes
1answer
147 views

Is Entropy = Information circular or trivial?

I have seen several "maximum entropy distributions" used in the mathematical and statistical literature, often with the justification that they are "minimally informed" beyond the assumptions and data ...
7
votes
2answers
159 views

What is the most general formalism for machine learning?

Most of the literature I can find in the field of machine learning is extremely practical, listing many techniques you can use like neural networks, SVMs, random forests, and so on. There are lots of ...
6
votes
1answer
574 views

Trying to understand the basics of bayesian inference

This paper gives a somewhat gentle introduction to Bayesian inference: http://www.miketipping.com/papers/met-mlbayes.pdf I got to section 2.3 without much problems but got stuck in understanding that ...
6
votes
0answers
103 views

Normalizing factor for product of Gaussian densities - interpretation with Bayes theorem

The normalizing factor for the product of two multivariate Gaussian densities, $f(x)$ and $g(x)$ with mean vectors $a$ and $b$ respectively, and covariance matrices $A$ and $B$ respectively, is itself ...
5
votes
1answer
273 views

Is there an introduction to probability and statistics that balances frequentist and bayesian views?

Perhaps, roughly, I might be described as advanced undergraduate regarding mathematics. However, I have not learned statistics and have only learned elementary probability. Does there exist a book or ...
5
votes
1answer
115 views

hint with Bayes rule problem

The pirate Captain Queequeg has a lazy crew and suspects they are planning to stage a mutiny. Captain Queequeg's solution is to have every member of the crew roll Queequeg's lucky die. If the roll is ...
4
votes
2answers
3k views

Why is $P(X,Y|Z)=P(Y|X,Z)P(X|Z)$?

Could anyone derive or explain why the formula $P(X,Y|Z)=P(Y|X,Z)P(X|Z)$ is true? I understand conditional probability definition, but this formula confuses me and makes my head hurt x) Here's ...
4
votes
3answers
529 views

Why does Bayes' theorem work?

Why does Bayes' theorem work? I'm not looking for a cryptic math demonstration. Rather, what I'm interested in is the intuition behind the theorem that allows to obtain the a posteriori probability ...
4
votes
3answers
152 views

Facebook Question (Data Science)

Out of curiosity, here's a question from Glassdoor (Facebook Data Science Interview) You're about to get on a plane to Seattle. You want to know if you should bring an umbrella. You call 3 ...
4
votes
1answer
62 views

Bayes' Net Conditional Probability

I have a Bayes' Net with 4 boolean nodes connected in a diamond shape. I want to find the probability of one of the middle nodes being true given that the ones above and below are both true. So ...
4
votes
1answer
1k views

Bayesian posterior with truncated normal prior

Suppose we observe one draw from the random variable $X$, which is distributed with normal distribution $\mathcal{N}(\mu,\sigma^2)$. The variance $\sigma^2$ is known, $\mu$ isn't. We want to estimate ...
4
votes
3answers
131 views

Is there any research field dedicated to estimating a “game” itself in game theory?

Game theory stuffs usually provide how a "game" works and then tries to figure out solutions - but I am wondering if there is any research field dedicated to estimating the full rules of a game. So ...
4
votes
1answer
116 views

Bayesian Updating with 1 Signal but 2 Unknowns

Suppose I have an unknown variable $X_i = \alpha_i + \beta_i$ where $\alpha$ is one of 2 different values {${\alpha_1, \alpha_2}$} such that $\alpha = \alpha_1$ with probability $p_1$ and $\beta$ is ...
4
votes
2answers
59 views

What is an approach for optimizing the values of a matrix?

My apologies if I get some terminology wrong, I don't have a formal math background; half my problem is articulating what I'm trying to do and identifying the domain of math that deals with this kind ...
4
votes
2answers
695 views

Application of Bayes Theorem

I am reading Nate Silver's book "The Signal and the Noise" and have a question about Bayes Theorem. I've created my own example and am trying to wrap my mind around the conclusion. Let's say, before ...
4
votes
1answer
140 views

Does Bayesian probability have a different interpretation of a random variable?

Bayesian probability interprets the meaning of the probability of a random variable as some degree of belief. But does this result in any difference in the interpretation of a random variable itself? ...
4
votes
0answers
53 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
4
votes
1answer
94 views

Bayesian inference

I'm a bit confused with arranging the Bayes equation to update probability. Say, I have the following data: $P(\text{blue birds in the whole study area}) = 0.16$; $P(\text{all except blue colored ...
3
votes
4answers
113 views

Bayes, two tests in a row

I came up with a standard Bayesian example as to point out my confusion. There is an epidemic. A person has a probability $\frac{1}{100}$ to have the disease. The authorities decide to test the ...
3
votes
1answer
104 views

Bayes factor and Posterior odds

Consider the following posterior odds \begin{equation*} \frac{P(H|D_1,D_2)}{P(\overline{H}|D_1,D_2)}=\frac{P(D_2|H,D_1)\times P(D_1|H)P(H)}{P(D_2|\overline{H},D_1)\times ...
3
votes
2answers
85 views

A house is guarded by two alarms

I am trying to wrap my head around the following problem A house is guarded by two alarms. If Alarm 1 fires, p(theft) = 80% If Alarm 2 fires, p(theft) = 70% If both alarms fire at the same time, ...
3
votes
2answers
953 views

Why would I use Bayes' Theorem if I can directly compute the posterior probability?

I fully understand the mechanics of Bayes' Theorem. However, I am wondering when do I need to use it? If I am able to compute the posterior probability directly from measured data, why would I need to ...
3
votes
2answers
196 views

Differentiating the posterior distribution function

I am learning about Bayesian statistics and I'm currently doing loss functions. Let $f(\theta | \mathbf{x} ) $ be a posterior pdf . Let $F(\theta | \mathbf{x} ) $ be the associated distribution ...
3
votes
1answer
1k views

What is the extension of Bayesian Network into cyclic graph?

The wikipage of Bayesian Network says "Formally, Bayesian networks are directed acyclic graphs whose nodes represent random variables in the Bayesian sense" But in the model I need to build, cyclic ...
3
votes
1answer
147 views

How to do Bayesian updating on biased information?

You have a coin that you can flip, but you can't see. It's a weighted $3$-sided coin taken (uniformly) randomly from some small known collection of $100$ weighted coins. However, we don't know how ...
3
votes
1answer
265 views

What is the meaning of “mean-field”?

In lots of Bayesian papers, people use variational approximation. In lots of them they call it "mean-field variational approximation". Does anyone know what is the meaning of mean-field in this ...
3
votes
1answer
577 views

Bayesian Inference in Measure Theory

What's the deal. How does this work, or can you point me to some references? I tried $\mu(A|B) = \mu(A \cap B) / \mu(B)$ and got stuck on $\mu(B) = 0$. Edit: Sorry for being lazy. My background is ...
3
votes
1answer
370 views

How do I calculate the aposteriori probability distribution for someone's answer to a poll being an approval?

Imagine I'm polling a random sample from the population and it asks them if they approve of the President or not. I also ask them some categorical demographic questions (age-bracket, race, gender, ...
3
votes
2answers
67 views

How can Bayesian and Frequentist approach be different?

Let's say I am trying to add numbers, like say one to ten. I can either add them in order, or I can notice that I can group them into five groups of eleven, so I suppose which method to use depends on ...
3
votes
2answers
69 views

Poker odds: Chances of a straight flush, given H4,H5

I'm trying to learn Bayes's formula, and am coming up with some poker problems to learn this. My problem is as following: given a $H4,H5$ ($4$ of hearts, $5$ of hearts) hand, what are the odds that ...
3
votes
2answers
90 views

Conjugate priors make calculations easier but at what cost to the model?

As I understand, when we have a parametric pdf and need to estimate the parameter based on some observed fact, we tend to choose a conjugate prior of the pdf for the parameter. Because conjugate prior ...
3
votes
1answer
473 views

Computing posterior distribution for AR(1) model

Question: For this question, note that the notation $y_{1:T} = (y_1, y_2, \cdots, y_T)$, ie, a vector of random variables. Consider the following AR(1) model: \begin{align*} y_{t+1} = \phi y_t + ...
3
votes
1answer
287 views

Maximum Entropy Distribution When Mean and Variance are Not Fixed with Positive Support

I know when the mean and variance of $\ln x$ are both fixed, then the maximum entropy probability distribution is lognormal. When the mean of a random variable is fixed the MEPD is the exponential ...
3
votes
1answer
63 views

Implied prior with relationship $y=\text{arccot}(x)$

I'm trying to solve an exercise, which I think I have almost managed to solve but not quite. Any help would be appreciated! So, what we have is a vector which we obtain by norming the vector ...
3
votes
1answer
215 views

What's the difference between Maximum a posteriori and Bayes' rule?

What's the difference between Maximum a posteriori and Bayes' rule? They look similar, except that I do understand Bayes' rule and I don't understand MAP. The people I asked - who work in math and ...
3
votes
1answer
101 views

Is there a formal explanation of the concept of “improper prior” in Bayesian statistics?

The Bayesian concept of "improper prior" seems to be surrounded with magic. Even formal, Bayesian-oriented books, such as Schervish's "Theory of Statistics", treat it with the heuristic hand waving ...
3
votes
0answers
52 views

What is the math behind calling election seats with confidence, before all votes have been counted?

On election night, predictions are made on the winner of each district, after only a fraction of the vote has been counted up. How is this done? Say there is a seat up for election, and 10,000 votes ...
3
votes
0answers
58 views

Fredholm Integral in Bayesian Appliation

Let $X = x_1, x_2, \ldots, x_n$ be a sequence of Bernoulli random variables with $k$ successes. Suppose that, given $X$, the posterior predictive probability of $x_{n+1} = x$ is known to be $g(x)$ ...
3
votes
0answers
27 views

If $P(B\text{ }|\text{ }A)=1-\epsilon$ and $P(C\text{ }|\text{ }B)=1$ then $P(C\text{ }|\text{ }A)\geq 1-\epsilon$ [duplicate]

If $$1=P(C\text{ }|\text{ }B)=\frac{P(C\cap B)}{P(B)}$$ then we know that $P(C\cap B)=P(B)$. If $P(B\text{ }|\text{ }A)=1-\epsilon$ for $\epsilon\geq 0$ then $$P(A)=\frac{P(B\cap ...
3
votes
1answer
104 views

Bayesian formula for weather exercise

If it is nice weather on one day, the probability that it is going to be nice again the next is $13/15$. If it is raining on one day, the prob. that it is going to be raining again the next day is ...
3
votes
1answer
172 views

Questions on Bayesian analysis of an opinion poll (an example in a book)

I'm sorry in advance for rather long questions. This is an example in "Bayesian logical data analysis for physical sciences" by P. C. Gregory and I have some questions about the example. In a poll ...
3
votes
0answers
115 views

Is there a name in the literature for a projectivized measure?

By a projectivized measure I mean a nonzero measure on some measurable space $X$ up to scaling. If a nonzero measure is finite, its projectivization can be identified with its normalization (to have ...
2
votes
6answers
251 views

What is the probability of the box?

Your box of cereal may be a contest winner! It's rattling, which 100% of winning boxes do. Of course 1% of all boxes rattle and only one box in a million is a winner. What is the probability that your ...
2
votes
1answer
79 views

What is the probability of two or more from n events occuring?

A number of independent events, say $A$, $B$, $\ldots\,$, $E$, can happen with associated probabilities $P(A)$, $P(B)$, $\ldots$ For each event that happens I have to pay £10. The likelihood I have ...
2
votes
1answer
368 views

I am confused about Bayes' rule in MCMC

Bayes' rule appears to bevery simple at first sight, but when studied deeply I find it is difficult and confusing, especially in MCMC applications when multiple parameters need to be estimated. For ...
2
votes
3answers
517 views

13 DOF Kalman filter

I'm trying to develop a system with the following characteristics: Inputs: 3-axis accelerometer [3 DOF] 3-axis gyroscope [3 DOF] GPS with three parameters (lat, lon, altitude) [3 DOF] Barometric ...
2
votes
2answers
58 views

How to find out number of possible outcomes by trying over and over?

While working on my network exploration tool project, I've ran across the problem of reliably determining number of possible exit addresses of a tunnel with single entrance. I've came up with ...
2
votes
3answers
155 views

Questions about Bayesian inference

From Wikipedia The prior distribution is the distribution of the parameter(s) before any data is observed, i.e. $p(\theta \mid \alpha )$. ... The sampling distribution is the distribution of ...
2
votes
2answers
105 views

In a deck of cards, if the second card picked is a heart, what is the probability that the first card picked was a heart?

Assume its a deck of 8 cards with 2 cards of each suit. My analysis is: A = First card is heart B = Second card is heart P(A) = 1/4 P(B) = 1/4 P(B|A) = 1/7 P(A|B) = P(A) * P(B|A) / P(B) = 1/4 ...