The approach and interpretation of probability associated with Bayes theorem; usually used as opposed to the frequentist approach. It can be seen as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain. A Bayesian probabilist starts with some prior ...

learn more… | top users | synonyms

6
votes
0answers
136 views

Normalizing factor for product of Gaussian densities - interpretation with Bayes theorem

The normalizing factor for the product of two multivariate Gaussian densities, $f(x)$ and $g(x)$ with mean vectors $a$ and $b$ respectively, and covariance matrices $A$ and $B$ respectively, is itself ...
5
votes
0answers
106 views

Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ ...
5
votes
0answers
221 views

Bartlett's paradox in Bayesian evidence

I've come across Bartlett's "paradox" (not to be confused with Lindley's paradox, also known as the Lindley-Bartlett paradox) in Bayesian statistics. The paradox originates from Bartlett's 1957 paper, ...
4
votes
0answers
65 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
4
votes
0answers
63 views

Fredholm Integral in Bayesian Appliation

Let $X = x_1, x_2, \ldots, x_n$ be a sequence of Bernoulli random variables with $k$ successes. Suppose that, given $X$, the posterior predictive probability of $x_{n+1} = x$ is known to be $g(x)$ ...
3
votes
0answers
61 views

What is the math behind calling election seats with confidence, before all votes have been counted?

On election night, predictions are made on the winner of each district, after only a fraction of the vote has been counted up. How is this done? Say there is a seat up for election, and 10,000 votes ...
3
votes
0answers
122 views

Is there a name in the literature for a projectivized measure?

By a projectivized measure I mean a nonzero measure on some measurable space $X$ up to scaling. If a nonzero measure is finite, its projectivization can be identified with its normalization (to have ...
2
votes
0answers
31 views

Example of Kalman filter with irregular time steps/weights to observations

I am trying to find a Kalman filter implementation which does not make the assumption that each observation is equally spaced and of equal weight. In particular, I have measurements of a process ...
2
votes
0answers
27 views

a variant of MLE of a normal distribution

It is well-known that if we have "n" sample observations from normal distribution with unknown mean, then the sample mean would be the MLE for the mean of the normal distribution. However, let's ...
2
votes
0answers
96 views

Correlation of belief distributions from distinct signals

Anne and Bob are two Bayesians who initially share a non-degenerate prior about a binary state of the world. Anne observes some signal (i.e., an experiment in Blackwell's terminology) about the state ...
2
votes
0answers
46 views

How can I infer order from partially ordered discrete sequences?

A really interesting problem that I can't stop thinking about! Have run in to this a couple of times but yet to find a smart approach to either solve or frame this problem. This is my try at ...
2
votes
0answers
62 views

In what sense is the Bayesian posterior mean a “convex combination”?

This is related to a previous question that hasn't gotten an answer: Definition of convex combination with matrix-vector multiplication Suppose I want to estimate $x \in \mathbb{R}^n$ from two ...
2
votes
0answers
143 views

Bayesian Shrinkage Factor

Vasicek(1973), referenced in this paper(See bottom of page 16) explains a method of shrinking individual betas $\beta^{TS}$ toward a cross-sectional mean $\beta^{XS}$ as follows: for each time $t$, ...
2
votes
0answers
36 views

Improper Lebesgue prior normalization in Bayesian filtering

Suppose we have a conditionally Gaussian Linear State Space Model (CGLSSM) where $Y_t=(X_t,S_t)_{t \in \mathbb{N}}$ is the Markov chain of hidden states, where for each $t \in \mathbb{N}$, $S_t \in ...
2
votes
0answers
77 views

Optimal Stopping for One-Armed Bandit with a Fixed, Known Payout.

I'm very new to bandit problems (apologies if I've formatted my question incorrectly), but I have to solve the optimal stopping of what I think is a very simple case. I have a bandit problem with one ...
2
votes
0answers
68 views

Bayesian Chain rule

I am going thorugh a Naive Bayes Classifier, and faced the following: $p(y|a,b,c) = \frac{p(a|y,b)*p(y|c)}{p(a|b,c)}$ When I am trying to derive the above, these are my steps: ...
2
votes
0answers
50 views

Does this question work with Bayes formula?

Looking at slide 11, Example 1.10 from: http://www-users.aston.ac.uk/~cornford/probmod/ProbMod310810_Ch1.pdf Luke has been told he’s lucky and has won a prize in the lottery. There are 5 prizes ...
2
votes
0answers
32 views

Is this problem suited for Bayesian inference?

Suppose that the quality of a widget is distributed according to a score, given by a normal distribution with mean 1 and variance σ^2. A fraction, π of all widgets are defective. The cost of having an ...
2
votes
0answers
68 views

Generalized Bayes Estimator

Consider a decision problem in which the model parameter, $\theta$, is any integer, the distribution for the integer observation, y, given $\theta$ is $P(y|\theta) = 1/3$ if $y \in [\theta - 1, \theta ...
2
votes
0answers
174 views

Posterior predictive distribution in a Bernoulli process.

Suppose there are $k$ successes in a Bernoulli population $ X = \{x_1, \ldots, x_n\}$. I would like to calculate the posterior predictive distribution $f(x | X)$ where $x = \{0,1\}$. I assume the ...
2
votes
0answers
115 views

Maximum Posterior: $ p(\bf{w}\mid\bf{x},\bf{t},\alpha,\beta) \propto p(\bf{t}\mid\bf{x},\bf{w},\beta)p(\bf{w}\mid\alpha) $ for Gaussian Distribution

At the moment I take a look at the book Pattern Recognition and Machine Learning from Christopher Bishop and as I try to understand the basics of the probability theory I get stuck trying to ...
2
votes
0answers
151 views

Building Bayesian Networks, Causality and Cyclic Reasoning

I am studying Bayesian Statistics and I am trying to get a good understanding on Bayesian Networks, which seems to be vital in order to make something useful in Machine Learning. Most of the texts I ...
2
votes
0answers
57 views

Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
2
votes
0answers
103 views

Probability distribution for a digit of a number

If someone choose a digit $\alpha$ and a digit $\beta$ independently. Each one can be in $0,1, ...,9$. So $\mu = \alpha \beta$ (e.g. if $\alpha = 5$ and $\beta = 3$ then $\mu =53$). And I observe a ...
2
votes
0answers
72 views

Gaussian Bayesian filtering with bound observation ($b_1<x<b_2$)

Suppose we have a Normal r.v $$ x \sim \mathcal{N}(\mu, \sigma^2) $$ and a Normal prior of $\mu$ $$ \mu \sim \mathcal{N}(\theta, \delta^2) $$ I know how to do the Bayesian update with a ...
1
vote
0answers
16 views

Expected utility of action, given probability model

We record measurements of an appartus every day. If apparatus doesn't break (it has probability equal to $1-p_2$), it will measure zero with probability $p_1$. If apparatus breaks (probability $p2$), ...
1
vote
0answers
29 views

Find a conditional probability of a Bayes' Net knowing only the prior probability of the root.

Given three nodes A,B,C that form a Bayes Network as the following: (A)-->(B)-->(C) If we know the prior probability of A is 0.3, i.e. P(A)=0.3, is this ...
1
vote
0answers
14 views

Posterior for Beta Binomial Distribution with Repeated Observations

I'm working on a question with simultaneous learning about an underlying population and individual members of the population. The basic setup is: Let $N_g$ be the size of a population. At any point ...
1
vote
0answers
31 views

Can the parameter of prior probability depends on data?

In Bayseian approach https://en.wikipedia.org/wiki/Prior_probability we often use prior probability. Can we have a prior probability distribution with parameters and while estimating the posterior ...
1
vote
0answers
26 views

Bayesian Gaussian Mixture model

I am trying to fit basic Gaussian mixture with a Bayesian model. My likelihood function is Gaussian, with std=1, and the only parameter is the mean, chosen from $\{0,1,\dots,14,15\}$ and my prior is ...
1
vote
0answers
25 views

Choosing a non-convex global optimization algorithm based on the number of permitted steps

Can anyone comment on the most suitable approach for the following optimization problem: We are given finite bounds for a set of $n$ real-valued parameters of an unknown deterministic function. The ...
1
vote
0answers
13 views

Denominator in Maximum Posterior Estimation - How to Interpret?

Suppose we're given a sequence $x_1,\ldots,x_n$ of realizations of i.i.d. $\mathcal{N}(\mu,\sigma^2)$ random variables and we want to apply maximum posterior estimation to estimate the parameters ...
1
vote
0answers
34 views

How do I solve a under-determined quadratic multi-variate system?

I have the following equation: $$ Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_3 + \beta_{11} X_{1}^2 + \beta_{22} X_{2}^2 + \beta_{33} X_{3}^2 + \beta_{12} X_{1} X_{2} + \beta_{23} X_{2} ...
1
vote
0answers
24 views

Markov-Chain Monte-Carlo: Are transformations on the inputs valid?

The problem: I am trying to solve a high dimensional (up to ~50) class of data fitting & modelling problems. The user specifies the problem, so I would like to make the configuration as easy as ...
1
vote
0answers
48 views

Confusion with Bayesian Linear Regression

In the book Gaussian Processes for Machine Learning in Chapter 2 p. 11 (see http://www.gaussianprocess.org/gpml/chapters/RW2.pdf), eq. 2.9 states: $p(f_* | X, y) = \int p(f_* | x_*,w) p(w|X, y)dw$ ...
1
vote
0answers
36 views

Distribution of unknown covariance matrix, given variance of linear combination

Suppose I am uncertain about the covariance of a vector-valued random variable $X$, but the variance of some linear combination is known. How does this affect the distribution of $X$? Specifically ...
1
vote
0answers
14 views

Working out closed form of shifted poisson distribution

In the article "Bayesian variable selection for Poisson regression with underreported responses" the author defines $t_i^0$ as the number of actual occurences in a study in the $i$th covariate ...
1
vote
0answers
160 views

distribution of the length for a random walk on an infinite 2D grid

In connection with the flatland paradox, consider a 2D-random walk $(X_n)$ on $\mathbb{Z}^2$: the four moves of length one to W,E,N, and S are equaly likely at each time. For a fixed number of moves ...
1
vote
0answers
22 views

Evaluating an expected value in Jeffrey's prior for binomial distribution

The material I'm reading derives Jeffrey's prior (or rather, the Fisher information for the Jeffrey's) for single-parameter binomial distribution in a manner quite similar to this Wikipedia article. ...
1
vote
0answers
34 views

Under what assumptions is the following first moment monotone?

I'm working on an economic model and am encountering the following mathematical issue. Let $x\sim \mathcal{N}(\mu,1)$, and define $$V(\mu)=\int_0^{\hat x(\mu)}x ...
1
vote
0answers
59 views

Log likelihood function for binary classification

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes ...
1
vote
0answers
53 views

Baye's Classifier for recovering a signal from a measurement

Below is the question i am having trouble with: Independent and identically distributed symbols s(n) = ±1 are transmitted over the channel C(z) = 1 + z −1 . Symbols s(n) = +1 occur with probability p ...
1
vote
0answers
63 views

Evidence propagation in bayesian network

I'm currently trying to wrap my head around evidence propagation in bayesian network (simple tree propagation) but I'm having trouble finding information about the process. As an example, let's take ...
1
vote
0answers
38 views

complicated posterior distribution

I have a question concerning a rather specific posterior. It should be a simple application of Bayes' Theorem. However, I am always confused here. I try my best to describe the situation. There are ...
1
vote
0answers
60 views

Rate of convergence of Bayesian posterior

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let ...
1
vote
0answers
23 views

Bayesian shrinkage doesn't affect eigenspace

The book Machine Learning a Probabilistic Perspective by Kevin Murphy on page 130 states following fact without proof: Consider the MLE estimate of covariance matrix $\Sigma_{\text{MLE}}$. The ...
1
vote
0answers
194 views

Explanation of the TrueSkill bayesian ranking algorithm in a two-person game, like Tennis?

TrueSkill is mostly used for ranking and matching players on Xbox Online Games, it is a general rating model that could be applied to any game, including Chess, Tennis or Football. It models every ...
1
vote
0answers
27 views

Which is a good book to read about convergence of posterior measure?

I am working on Bayesian statistics and would like to know about a good text book about convergence of posterior measure.
1
vote
0answers
319 views

About Bayesian formula and rating system

I'm building a scoring system with score from 0 to 5) and I would like to sort products according to the number of reviews and their scores. After some research on the Internet I have found two ...
1
vote
0answers
39 views

Recursive Bayesian Estimation, $p(C_k|x)$ as likelihood

I''ve been struggeling with this problem for the last couple of days. The main goal is to use the probabilistic classification output $p(C_k|x)$, from for example a logistic regression, to enhance ...