# Tagged Questions

The approach and interpretation of probability associated with Bayes theorem; usually used as opposed to the frequentist approach. It can be seen as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain. A Bayesian probabilist starts with some prior ...

136 views

### Normalizing factor for product of Gaussian densities - interpretation with Bayes theorem

The normalizing factor for the product of two multivariate Gaussian densities, $f(x)$ and $g(x)$ with mean vectors $a$ and $b$ respectively, and covariance matrices $A$ and $B$ respectively, is itself ...
106 views

### Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ ...
221 views

### Bartlett's paradox in Bayesian evidence

I've come across Bartlett's "paradox" (not to be confused with Lindley's paradox, also known as the Lindley-Bartlett paradox) in Bayesian statistics. The paradox originates from Bartlett's 1957 paper, ...
65 views

### Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
63 views

### Fredholm Integral in Bayesian Appliation

Let $X = x_1, x_2, \ldots, x_n$ be a sequence of Bernoulli random variables with $k$ successes. Suppose that, given $X$, the posterior predictive probability of $x_{n+1} = x$ is known to be $g(x)$ ...
61 views

### What is the math behind calling election seats with confidence, before all votes have been counted?

On election night, predictions are made on the winner of each district, after only a fraction of the vote has been counted up. How is this done? Say there is a seat up for election, and 10,000 votes ...
122 views

### Is there a name in the literature for a projectivized measure?

By a projectivized measure I mean a nonzero measure on some measurable space $X$ up to scaling. If a nonzero measure is finite, its projectivization can be identified with its normalization (to have ...
31 views

### Example of Kalman filter with irregular time steps/weights to observations

I am trying to find a Kalman filter implementation which does not make the assumption that each observation is equally spaced and of equal weight. In particular, I have measurements of a process ...
27 views

### a variant of MLE of a normal distribution

It is well-known that if we have "n" sample observations from normal distribution with unknown mean, then the sample mean would be the MLE for the mean of the normal distribution. However, let's ...
96 views

### Correlation of belief distributions from distinct signals

Anne and Bob are two Bayesians who initially share a non-degenerate prior about a binary state of the world. Anne observes some signal (i.e., an experiment in Blackwell's terminology) about the state ...
46 views

### How can I infer order from partially ordered discrete sequences?

A really interesting problem that I can't stop thinking about! Have run in to this a couple of times but yet to find a smart approach to either solve or frame this problem. This is my try at ...
62 views

### In what sense is the Bayesian posterior mean a “convex combination”?

This is related to a previous question that hasn't gotten an answer: Definition of convex combination with matrix-vector multiplication Suppose I want to estimate $x \in \mathbb{R}^n$ from two ...
143 views

### Bayesian Shrinkage Factor

Vasicek(1973), referenced in this paper(See bottom of page 16) explains a method of shrinking individual betas $\beta^{TS}$ toward a cross-sectional mean $\beta^{XS}$ as follows: for each time $t$, ...
36 views

174 views

### Posterior predictive distribution in a Bernoulli process.

Suppose there are $k$ successes in a Bernoulli population $X = \{x_1, \ldots, x_n\}$. I would like to calculate the posterior predictive distribution $f(x | X)$ where $x = \{0,1\}$. I assume the ...
115 views

### Maximum Posterior: $p(\bf{w}\mid\bf{x},\bf{t},\alpha,\beta) \propto p(\bf{t}\mid\bf{x},\bf{w},\beta)p(\bf{w}\mid\alpha)$ for Gaussian Distribution

At the moment I take a look at the book Pattern Recognition and Machine Learning from Christopher Bishop and as I try to understand the basics of the probability theory I get stuck trying to ...
151 views

### Building Bayesian Networks, Causality and Cyclic Reasoning

I am studying Bayesian Statistics and I am trying to get a good understanding on Bayesian Networks, which seems to be vital in order to make something useful in Machine Learning. Most of the texts I ...
57 views

### Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
103 views

### Probability distribution for a digit of a number

If someone choose a digit $\alpha$ and a digit $\beta$ independently. Each one can be in $0,1, ...,9$. So $\mu = \alpha \beta$ (e.g. if $\alpha = 5$ and $\beta = 3$ then $\mu =53$). And I observe a ...
72 views

### Gaussian Bayesian filtering with bound observation ($b_1<x<b_2$)

Suppose we have a Normal r.v $$x \sim \mathcal{N}(\mu, \sigma^2)$$ and a Normal prior of $\mu$ $$\mu \sim \mathcal{N}(\theta, \delta^2)$$ I know how to do the Bayesian update with a ...
16 views

### Expected utility of action, given probability model

We record measurements of an appartus every day. If apparatus doesn't break (it has probability equal to $1-p_2$), it will measure zero with probability $p_1$. If apparatus breaks (probability $p2$), ...
29 views

### Find a conditional probability of a Bayes' Net knowing only the prior probability of the root.

Given three nodes A,B,C that form a Bayes Network as the following: (A)-->(B)-->(C) If we know the prior probability of A is 0.3, i.e. P(A)=0.3, is this ...
14 views

### Posterior for Beta Binomial Distribution with Repeated Observations

I'm working on a question with simultaneous learning about an underlying population and individual members of the population. The basic setup is: Let $N_g$ be the size of a population. At any point ...
31 views

### Can the parameter of prior probability depends on data?

In Bayseian approach https://en.wikipedia.org/wiki/Prior_probability we often use prior probability. Can we have a prior probability distribution with parameters and while estimating the posterior ...
26 views

### Bayesian Gaussian Mixture model

I am trying to fit basic Gaussian mixture with a Bayesian model. My likelihood function is Gaussian, with std=1, and the only parameter is the mean, chosen from $\{0,1,\dots,14,15\}$ and my prior is ...
25 views

### Choosing a non-convex global optimization algorithm based on the number of permitted steps

Can anyone comment on the most suitable approach for the following optimization problem: We are given finite bounds for a set of $n$ real-valued parameters of an unknown deterministic function. The ...
13 views

### Denominator in Maximum Posterior Estimation - How to Interpret?

Suppose we're given a sequence $x_1,\ldots,x_n$ of realizations of i.i.d. $\mathcal{N}(\mu,\sigma^2)$ random variables and we want to apply maximum posterior estimation to estimate the parameters ...
34 views

59 views

### Log likelihood function for binary classification

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes ...
53 views

### Baye's Classifier for recovering a signal from a measurement

Below is the question i am having trouble with: Independent and identically distributed symbols s(n) = ±1 are transmitted over the channel C(z) = 1 + z −1 . Symbols s(n) = +1 occur with probability p ...
63 views

### Evidence propagation in bayesian network

I'm currently trying to wrap my head around evidence propagation in bayesian network (simple tree propagation) but I'm having trouble finding information about the process. As an example, let's take ...
38 views

### complicated posterior distribution

I have a question concerning a rather specific posterior. It should be a simple application of Bayes' Theorem. However, I am always confused here. I try my best to describe the situation. There are ...
60 views

### Rate of convergence of Bayesian posterior

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let ...
23 views

### Bayesian shrinkage doesn't affect eigenspace

The book Machine Learning a Probabilistic Perspective by Kevin Murphy on page 130 states following fact without proof: Consider the MLE estimate of covariance matrix $\Sigma_{\text{MLE}}$. The ...
194 views

### Explanation of the TrueSkill bayesian ranking algorithm in a two-person game, like Tennis?

TrueSkill is mostly used for ranking and matching players on Xbox Online Games, it is a general rating model that could be applied to any game, including Chess, Tennis or Football. It models every ...
27 views

### Which is a good book to read about convergence of posterior measure?

I am working on Bayesian statistics and would like to know about a good text book about convergence of posterior measure.
### Recursive Bayesian Estimation, $p(C_k|x)$ as likelihood
I''ve been struggeling with this problem for the last couple of days. The main goal is to use the probabilistic classification output $p(C_k|x)$, from for example a logistic regression, to enhance ...