The approach and interpretation of probability associated with Bayes theorem; usually used as opposed to the frequentist approach. It can be seen as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain. A Bayesian probabilist starts with some prior ...

learn more… | top users | synonyms

12
votes
6answers
6k views

Bayes Theorem Example in Nate Silver's The Signal and the Noise

In his book The Signal and the Noise, Nate Silver presents this example application of Bayes's Theorem on pp. 247-248: Consider a somber example: the September 11 attacks. Most of us would have ...
9
votes
1answer
181 views

Is Entropy = Information circular or trivial?

I have seen several "maximum entropy distributions" used in the mathematical and statistical literature, often with the justification that they are "minimally informed" beyond the assumptions and data ...
8
votes
1answer
57 views

Probability that a clumsy boy eats $k$ out of 20 candies

A week or two (or maybe more) ago, the following question was posted and then deleted just as I was getting to the end of my solution. Unfortunately I have now forgotten what my solution was going to ...
8
votes
2answers
204 views

What is the most general formalism for machine learning?

Most of the literature I can find in the field of machine learning is extremely practical, listing many techniques you can use like neural networks, SVMs, random forests, and so on. There are lots of ...
7
votes
2answers
11k views

Bayes rule with multiple conditions

I am wondering how I would apply Bayes rule to expand an expression with multiple variables on either side of the conditioning bar. In another forum post, for example, I read that you could expand $P(...
6
votes
1answer
349 views

Is there an introduction to probability and statistics that balances frequentist and bayesian views?

Perhaps, roughly, I might be described as advanced undergraduate regarding mathematics. However, I have not learned statistics and have only learned elementary probability. Does there exist a book or ...
6
votes
1answer
645 views

Trying to understand the basics of bayesian inference

This paper gives a somewhat gentle introduction to Bayesian inference: http://www.miketipping.com/papers/met-mlbayes.pdf I got to section 2.3 without much problems but got stuck in understanding that ...
6
votes
0answers
81 views

Probability of number of people who know a rumor

Suppose that among a group of $n$ people, some unknown number of people $K$ know a rumor. If someone knows the rumor, there is a probability $p$ that they will tell it to us if we ask. If they don't ...
6
votes
0answers
166 views

Normalizing factor for product of Gaussian densities - interpretation with Bayes theorem

The normalizing factor for the product of two multivariate Gaussian densities, $f(x)$ and $g(x)$ with mean vectors $a$ and $b$ respectively, and covariance matrices $A$ and $B$ respectively, is itself ...
5
votes
4answers
2k views

Facebook Question (Data Science)

Out of curiosity, here's a question from Glassdoor (Facebook Data Science Interview) You're about to get on a plane to Seattle. You want to know if you should bring an umbrella. You call 3 ...
5
votes
1answer
71 views

Let. $X \sim \mathcal{U}(0,1)$. Given $X = x$, let $Y \sim \mathcal{U}(0,x)$. How can I calculate $\mathbb{E}(X|Y = y)$?

Suppose that $X$ is uniformly distributed over $[0,1]$. Now choose $X = x$ and let $Y$ be uniformly distributed over $[0,x]$. Is it possible for us to calculate the "expected value of $X$ given $Y = y$...
5
votes
1answer
274 views

Hillary Clinton's Iowa Caucus Coin Toss Wins and Bayesian Inference

In yesterday's Iowa Caucus, Hillary Clinton beat Bernie Sanders in six out of six tied counties by a coin-toss*. I believe we would have heard the uproar about it by now if this was somehow rigged in ...
5
votes
0answers
173 views

Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ $...
5
votes
0answers
333 views

Bartlett's paradox in Bayesian evidence

I've come across Bartlett's "paradox" (not to be confused with Lindley's paradox, also known as the Lindley-Bartlett paradox) in Bayesian statistics. The paradox originates from Bartlett's 1957 paper, ...
5
votes
1answer
141 views

hint with Bayes rule problem

The pirate Captain Queequeg has a lazy crew and suspects they are planning to stage a mutiny. Captain Queequeg's solution is to have every member of the crew roll Queequeg's lucky die. If the roll is ...
4
votes
4answers
398 views

Bayes, two tests in a row

I came up with a standard Bayesian example as to point out my confusion. There is an epidemic. A person has a probability $\frac{1}{100}$ to have the disease. The authorities decide to test the ...
4
votes
2answers
6k views

Why is $P(X,Y|Z)=P(Y|X,Z)P(X|Z)$?

Could anyone derive or explain why the formula $P(X,Y|Z)=P(Y|X,Z)P(X|Z)$ is true? I understand conditional probability definition, but this formula confuses me and makes my head hurt x) Here's ...
4
votes
1answer
2k views

Bayesian posterior with truncated normal prior

Suppose we observe one draw from the random variable $X$, which is distributed with normal distribution $\mathcal{N}(\mu,\sigma^2)$. The variance $\sigma^2$ is known, $\mu$ isn't. We want to estimate $...
4
votes
1answer
69 views

Bayes' Net Conditional Probability

I have a Bayes' Net with 4 boolean nodes connected in a diamond shape. I want to find the probability of one of the middle nodes being true given that the ones above and below are both true. So ...
4
votes
3answers
137 views

Is there any research field dedicated to estimating a “game” itself in game theory?

Game theory stuffs usually provide how a "game" works and then tries to figure out solutions - but I am wondering if there is any research field dedicated to estimating the full rules of a game. So ...
4
votes
1answer
155 views

Bayesian Updating with 1 Signal but 2 Unknowns

Suppose I have an unknown variable $X_i = \alpha_i + \beta_i$ where $\alpha$ is one of 2 different values {${\alpha_1, \alpha_2}$} such that $\alpha = \alpha_1$ with probability $p_1$ and $\beta$ is ...
4
votes
1answer
776 views

Computing posterior distribution for AR(1) model

Question: For this question, note that the notation $y_{1:T} = (y_1, y_2, \cdots, y_T)$, ie, a vector of random variables. Consider the following AR(1) model: \begin{align*} y_{t+1} = \phi y_t + \...
4
votes
2answers
64 views

What is an approach for optimizing the values of a matrix?

My apologies if I get some terminology wrong, I don't have a formal math background; half my problem is articulating what I'm trying to do and identifying the domain of math that deals with this kind ...
4
votes
1answer
402 views

What is the meaning of “mean-field”?

In lots of Bayesian papers, people use variational approximation. In lots of them they call it "mean-field variational approximation". Does anyone know what is the meaning of mean-field in this ...
4
votes
2answers
1k views

Application of Bayes Theorem

I am reading Nate Silver's book "The Signal and the Noise" and have a question about Bayes Theorem. I've created my own example and am trying to wrap my mind around the conclusion. Let's say, before ...
4
votes
1answer
146 views

Does Bayesian probability have a different interpretation of a random variable?

Bayesian probability interprets the meaning of the probability of a random variable as some degree of belief. But does this result in any difference in the interpretation of a random variable itself?
4
votes
0answers
75 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set $X=${...
4
votes
0answers
65 views

Fredholm Integral in Bayesian Appliation

Let $X = x_1, x_2, \ldots, x_n$ be a sequence of Bernoulli random variables with $k$ successes. Suppose that, given $X$, the posterior predictive probability of $x_{n+1} = x$ is known to be $g(x)$ ...
4
votes
1answer
104 views

Bayesian inference

I'm a bit confused with arranging the Bayes equation to update probability. Say, I have the following data: $P(\text{blue birds in the whole study area}) = 0.16$; $P(\text{all except blue colored ...
3
votes
2answers
1k views

Why would I use Bayes' Theorem if I can directly compute the posterior probability?

I fully understand the mechanics of Bayes' Theorem. However, I am wondering when do I need to use it? If I am able to compute the posterior probability directly from measured data, why would I need to ...
3
votes
1answer
111 views

Bayes factor and Posterior odds

Consider the following posterior odds \begin{equation*} \frac{P(H|D_1,D_2)}{P(\overline{H}|D_1,D_2)}=\frac{P(D_2|H,D_1)\times P(D_1|H)P(H)}{P(D_2|\overline{H},D_1)\times P(D_1|\overline{H})P(\overline{...
3
votes
2answers
92 views

A house is guarded by two alarms

I am trying to wrap my head around the following problem A house is guarded by two alarms. If Alarm 1 fires, p(theft) = 80% If Alarm 2 fires, p(theft) = 70% If both alarms fire at the same time, ...
3
votes
2answers
201 views

Differentiating the posterior distribution function

I am learning about Bayesian statistics and I'm currently doing loss functions. Let $f(\theta | \mathbf{x} ) $ be a posterior pdf . Let $F(\theta | \mathbf{x} ) $ be the associated distribution ...
3
votes
1answer
194 views

Jaynes' taxicab problem

I am currently reading Jaynes' Probability Theory, The Logic of Science and am still trying to absorb everything. On page 190, he poses the following intriguing question, paraphrased here. Suppose ...
3
votes
3answers
240 views

Probability of independent events $P(ab)=P(a)*P(b)$

I know there are two ways to say event $a$ and $b$ are independent: $P(a)*P(b)=P(ab)$ $P(a\mid b)=P(a)$ and I can derive one from the other with the Bayes Formula $P(a|b)=P(ab)/P(b)$. My question ...
3
votes
2answers
2k views

What is the extension of Bayesian Network into cyclic graph?

The wikipage of Bayesian Network says "Formally, Bayesian networks are directed acyclic graphs whose nodes represent random variables in the Bayesian sense" But in the model I need to build, cyclic ...
3
votes
1answer
45 views

How does the posterior of a dirac prior look like?

Edit for the Moderators: Should this question migrate to stats.stackexchange? I have a very basic question concerning updating from a prior to a posterior in bayesian statistics. Setting: I ...
3
votes
1answer
1k views

Bayesian versus Classical (frequentist) Statistics

Very often in text-books the comparison of Bayesian vs. Classical Statistics are presented upfront in a very abstract way. For example, in the current book I'm studying there's the following ...
3
votes
1answer
79 views

What am I doing wrong in calculating Fisher Information of Triangular Distribution?

I am trying to find Jeffrey's prior for the Triangular distribution which has the following probability density function: $$f(x\mid \theta) = \begin{cases} \dfrac{2x}{\theta} & : x \...
3
votes
1answer
153 views

Not sure how to solve Bayesian parameter learning problem

I could use some help solving a problem about a Dirichlet prior. We have a multinomial distribution over an alphabet of 27 symbols parameterized by $\mathbf{\theta}=(\theta_1, ..., \theta_{27})$. We ...
3
votes
1answer
173 views

How to do Bayesian updating on biased information?

You have a coin that you can flip, but you can't see. It's a weighted $3$-sided coin taken (uniformly) randomly from some small known collection of $100$ weighted coins. However, we don't know how ...
3
votes
1answer
867 views

Bayesian Inference in Measure Theory

What's the deal. How does this work, or can you point me to some references? I tried $\mu(A|B) = \mu(A \cap B) / \mu(B)$ and got stuck on $\mu(B) = 0$. Edit: Sorry for being lazy. My background is ...
3
votes
1answer
386 views

How do I calculate the aposteriori probability distribution for someone's answer to a poll being an approval?

Imagine I'm polling a random sample from the population and it asks them if they approve of the President or not. I also ask them some categorical demographic questions (age-bracket, race, gender, ...
3
votes
1answer
32 views

Calculate Conditional Probability for program that does not crash

I got these 2 questions in exam, but unfortunately i failed to solved these. 1) you want to buy a computer. The probability that you can run the probabilistic program $X$ on it is $97$% ...
3
votes
1answer
34 views

Determine how likely it is that a set of boolean data is produced by a distribution

Suppose we have a collection of independent Boolean random variables $X_i$ and $Y_i$ (for $1 \le i \le N$), and are told $p_i = P(X_i = 1)$ for all $i$. We are now given a set of values $x_i$ that was ...
3
votes
1answer
68 views

Difference between Frequentist and Bayesian approach to Statistics

What is the difference between the Frequentist vs. the Bayesian approach to Statistics? Would someone be so kind to come up with a simple example that shows how the approaches and possibly the ...
3
votes
2answers
75 views

How can Bayesian and Frequentist approach be different?

Let's say I am trying to add numbers, like say one to ten. I can either add them in order, or I can notice that I can group them into five groups of eleven, so I suppose which method to use depends on ...
3
votes
2answers
85 views

Poker odds: Chances of a straight flush, given H4,H5

I'm trying to learn Bayes's formula, and am coming up with some poker problems to learn this. My problem is as following: given a $H4,H5$ ($4$ of hearts, $5$ of hearts) hand, what are the odds that I'...
3
votes
2answers
95 views

Conjugate priors make calculations easier but at what cost to the model?

As I understand, when we have a parametric pdf and need to estimate the parameter based on some observed fact, we tend to choose a conjugate prior of the pdf for the parameter. Because conjugate prior ...
3
votes
1answer
377 views

Maximum Entropy Distribution When Mean and Variance are Not Fixed with Positive Support

I know when the mean and variance of $\ln x$ are both fixed, then the maximum entropy probability distribution is lognormal. When the mean of a random variable is fixed the MEPD is the exponential ...