Questions tagged [bayesian]
The approach and interpretation of probability associated with Bayes' theorem; usually used as opposed to the frequentist approach. It can be seen as an extension of logic that enables reasoning with propositions whose truth or falsity is uncertain. A Bayesian probabilist starts with some prior probability, and evaluates the evidence in favour of a hypothesis by combining the prior with the likelihood function of the observed data.
1,993
questions
0
votes
0
answers
10
views
Using Bayesian statistics in time series forecasting
I would like to forecast demand count time series of taxi fleets at different locations on the map at different points in time. I.e. multivariate demand Time series forecasting.
Given hierarchinal ...
- 247
0
votes
1
answer
24
views
Bayesian statistics - explanation of evidence
Despite trying to read multiple resources about Bayesian statistics, I cannot find a (free) resource which explains what is exactly $P(D)$. Most of the resources explain it somehow conceptually ...
- 239
-1
votes
0
answers
42
views
Monty Hall problem with no guarantee
There 's an 80% chance a ball is in a chest of drawers with 4 drawers.
If we open 3 and find they're empty ,what is the probability it s in the 4th (all drawers have an identical chance of having the ...
- 303
1
vote
1
answer
45
views
Deriving the expression for the posterior predictive distribution.
We have $Y \mid \Theta=\theta \sim \operatorname{Po}(\theta)$ and $\Theta \sim \operatorname{Gamma}\left(\alpha_0, \lambda_0\right)$, the expression for the posterior predictive distribution is ...
- 25
3
votes
2
answers
86
views
About the notation $\mathbb{E}_t [\text{d} s_t]$
My knowledge on stochastic calculus tells me that the notation $\text{d} X_t = \mu(X_t) \text{d} t + \sigma(X_t)\text{d} B_t$ is just an abbreviation of $X_t = X_0 + \int_0^t \mu(X_s) \text{d} s + \...
0
votes
1
answer
33
views
Show that the equivalence of MAP in Bayesian estimation to Structural Risk Minimization (SRM)
I'm sorry that I didn't explain my problem clearly😭I would like to add something.
I saw this problem in Machine Learning Method written by Li Hang(p16).
The book states the problem as below: "...
- 21
0
votes
1
answer
37
views
For any chaotic system such as a chaotic attractor does there exist a higher dimensional representation in which the system is no longer chaotic? [closed]
For a given n dimensional chaotic system such as a chaotic attractor or really a time dependent chaotic dynamic system does there exist a higher dimensional representation where the behavior is in ...
-1
votes
1
answer
69
views
Is there a better way to describe statistical "chance" on TV? [closed]
Something I've noticed a lot on TV is hosts/announcers using statistics incorrectly by converting past performances into present odds. For instance:
A football announcer says a kicker has "a 16%...
- 183
1
vote
1
answer
24
views
Marginal Posterior Distribution for Multinomial/Dirichlet Variables
Suppose that some data $(y_{1},\ldots,y_{J})$ are distributed multinomially with parameters $(\theta_{1},\ldots,\theta_{J})$ and that $\theta = (\theta_{1},\ldots,\theta_{J})$ has Dirichlet prior ...
- 1,194
2
votes
0
answers
20
views
+50
Confusion with Complex Gaussian process with Auto-covariance
I have a complex sequence $z(t)$ in time which I know to be a Gaussian process. I read that the complex Gaussian process is not only characterized by the covariance, but also the pseudo-covariance ...
- 107
0
votes
0
answers
36
views
How to prove $x^T\Sigma x$ is quadratic?
How to prove $x^T\Sigma x$ is quadratic? where $\Sigma$ is covariant varible(symmetric matrix)
I did simple calculation with 3x2 matrix , 2x2 covariance, 2x3 matrix. the result is 3x3 matrix. But how ...
- 367
1
vote
0
answers
25
views
Are we finding the density of $x$ or evaluating the density of $\theta$ at $x$? | Alpyadin Machine Learning
In section $4.4$ The Bayes Estimator of Alpaydin he discusses the use of the prior density of $p(\theta)$ to construct a posterior density for $\theta$. This is standard Bayesian estimation to get a ...
- 11
0
votes
0
answers
12
views
Getting negative predictions while fitting a Bayesian linear regression model [migrated]
I am trying to fit a Bayesian linear regression model on a data set of $3$ years. I have used both pymc and pytorch libraries and the NUTS sampler for sampling.
The dependent variable is Sales, and ...
- 1
0
votes
0
answers
23
views
What are the regularity conditions of the Bernstein-von Mises theorem in simple one-dimensional case?
I would like to apply the Bernstein-von Mises theorem in a very simple settings to be able to make the following claim:
Let $\theta \in [0,1]$ be a random parameter with strictly positive and bounded ...
- 1,530
1
vote
0
answers
15
views
The difference between the Bayesian estimator and MLE multiplied by $\sqrt{n}$ converges to zero.
Let $\theta$ be a random parameter with support $[0,1]$ and positive density, and let $X_1,X_2,\ldots \sim \rm N(\theta,1)$ be its i.i.d. observations. Define
$$ \delta_n := \sqrt{n} \Big(\hat \...
- 1,530
0
votes
0
answers
21
views
Expected squared difference between between the ML estimator and the posterior expectation.
Let $\theta$ be a random parameter with support $[0,1]$ and positive density, and let $X_1,X_2,\ldots ~ \rm N(\theta,1)$ be its i.i.d. observations. Does
$$ \rm E\Big[n\cdot \Big(\hat \theta_n(X_1,\...
- 1,530
0
votes
0
answers
24
views
Performing Maximum A Posteriori estimation on a set of dice results.
I have a set of data obtained from rolling a 20 sided dice 1000 times. I understand that ideally a dice would have a uniform distribution, and that forms my prior belief. But how exactly does one go ...
1
vote
1
answer
34
views
Deriving the posterior predictive mean for a Gaussian process with nonzero mean function
$\require{color}$
$\newcommand{\vect}[1]{{\mathbf{\boldsymbol{{#1}}}}}$
$\newcommand{\x}{\textbf{x}}$
$\newcommand{\y}{\textbf{y}}$
$\newcommand{\w}{\textbf{w}}$
$\newcommand{\wpriorcov}{\Sigma_p}$
$\...
- 8,507
0
votes
0
answers
20
views
Law of total probability conditional on normal distribution
Given two normal distribution $y_* | \mathbf{x}_*, \mathbf{w} \sim N( \mathbf{x}_*^T \mathbf{w}, \sigma_n^2)$ and $\mathbf{w} \sim N(\mathbf{\bar{w}}, A^{-1})$. I am trying to show that
$$y_* | \...
- 79
0
votes
0
answers
19
views
Bayesian updating with cyclically interdependent observations
Let $a,b,c$ be independent normally distributed variables with mean $0$ and standard deviation $\sigma_a,\sigma_b,\sigma_c$, respectively. What is the posterior distribution of the variables $a,b,c$ ...
- 1,530
0
votes
0
answers
17
views
Intuitive utility of the Jeffreys prior, eg. in Bernoulli trials
I understand the computation the Jeffreys prior, and also its historical motivation. I (somewhat) understand the theoretical desirability of a "prior-construction principle/method" that is ...
1
vote
0
answers
60
views
Is $\int_{0}^{1}\exp\left(-\frac{n(x-Ks)^2}{2Ks}\right)s^{\alpha-3/2}(1-s)^{\beta-1}\mathop{\mathrm ds}$ approximately Gaussian as $K\rightarrow 0$?
Suppose that $$X\mid p\sim\mathcal{N}\left(Kp,\frac{Kp}{n}\right)$$ for $K^2\approx 0$. Now, if $p\sim\text{Beta}(\alpha,\beta)$ for large $\alpha,\beta$ (say $\alpha\geq1.5$ and $\beta\geq 1$), what ...
- 68
2
votes
0
answers
32
views
Maximum entropy for continuous distributions, optimization problem
Reading E. T. Jaynes book. In chapter 12 he introduces an extension to Shannon's entropy for continuous distributions:
(*) $H_{I}^{c} = - \int \text{d}z \, p(z|I) \log{\frac{p(z|I)}{m(z)}}$
which has ...
- 21
0
votes
0
answers
20
views
Bayesian Updating with Conditional Independence of two tests
I have the following scenario of Bayes updating with which I struggle quite a bit.
Imagine we are interested in the probability that a given person has a disease $D$. We perform two different tests $...
- 11
2
votes
2
answers
85
views
How to understand the Posterior hyperparameters for Bernoulli in Beta conjugate prior?
From here: https://en.wikipedia.org/wiki/Conjugate_prior#When_the_likelihood_function_is_a_discrete_distribution
I know $\text{posterior} = \frac{\text{proir} \cdot \text{likelyhood}}{\text{evidence}}$...
- 367
0
votes
0
answers
17
views
Recursive formulas for the distributions in the state space model
I am having difficulty understanding the recursive formula for the posterior distribution $p(x_{0:t+1}|y_{1:t+1})$ of the state space model in which we assume $x_t$ is Markovian and $y_t$ are ...
- 5,023
0
votes
0
answers
20
views
Probability problem with uncertainty [duplicate]
In a City there are 2 Taxi companies: Blue and Green.
85% of the Taxis are Green, 15% of the taxis are Blue
A man has been hit by a taxi and he claims the taxi was blue but in tribunal the Judge ...
- 1
1
vote
2
answers
147
views
What is the probability that the man is guilty?
Problem
I try to build some connection between those text provided figure to formulate a bayes equation when I want to solve : "What is the probability that the man is guilty?"
I know that:
$...
- 367
0
votes
1
answer
57
views
Independence among random variables [closed]
Say we have the random variables $X_1$, $X_2$, $X_3$, $X_4$, $X_5$. We know that:
$X_5$ is influenced by $X_3$.
$X_4$ is influenced by both $X_2$ and $X_3$.
$X_2$ and $X_3$ are both influenced by $...
- 1,375
2
votes
0
answers
78
views
2 different answers, two different intuitions for Seattle raining probability problem, which one is correct? and why?
You are waiting for your flight to Seattle, and to pass the time you
call 3 friends in Seattle. You independently ask each one if it is
raining.
All 3 of your friends say “Yes, it is raining.”
But ...
- 43
-2
votes
1
answer
61
views
A brilliant introductory course on machine learning (mathematical perspective) (simulation + implementation) [closed]
I am a grad student with a relatively good understanding of stochastic analysis / probability theory, but only basic coding experience.
What is a good source (textbook or lecture notes) for an ...
- 422
2
votes
1
answer
77
views
If A,B indenpendent, and P(C|A),P(C|B) >P(C), what is relation P(C|A,B) with P(C)? [closed]
If A,B indenpendent, and $P(C|A),P(C|B) >P(C)$, what is relation $P(C|A,B)$ with $P(C)$?(> or <)
- 367
0
votes
0
answers
63
views
Exchangeability of y1 and y2.
I'm working through one of Gelman's exercises on exchangeability and am stuck on a seemingly simple exercise. We are given a box with N black and white balls but we not know how many of each. Task ...
- 35
0
votes
0
answers
7
views
Does likelihood function of your choice impact the asymptotic posterior of MCMC?
The metropolis Hasting algorithm decides whether to jump based on posterior probability. Namely, likelihood x prior.
Then it seems the density function that you choose (e.g., Poisson or Gaussian) can ...
- 23
0
votes
0
answers
12
views
Bayesian inference on Gaussian process without conjugate prior but particular prior distribution
I wondered if a well-known formula exists or if there is any reference I can look into for the following Bayesian inference problem.
An observed data point $y$ is a sum of true underlying value $x$ ...
0
votes
0
answers
49
views
Bayesian Learning: Finding the variance of signal
Suppose $x_i \sim N(10,4)$ - ie, the distribution is known.
There is a noisy signal $s_i \sim N(x_i, \sigma_e^2)$ and I want to estimate $\sigma_e$.
I see some pairs ($s_i, x_i$) but they are not '...
- 11
0
votes
1
answer
53
views
MAP estimation for conditional probability
I'm studying bayesian estimation of model parameters and i noticed in several ML books (Deep Learning Goodfellow, ML a probabilistic perspective K. Murphy) that they used the bayesian rule in the ...
0
votes
1
answer
41
views
Finding PDFs of conditional probability
$Y$ is a probability variable with only $0$ and $1$ as outcomes.
$$
P(Y = 0) = P(Y = 1) = \dfrac{1}{2}
$$
$f_{X|Y}(x|y)$ means the conditional probability function based on $Y$.
$$
Y = 0, X \sim N(...
- 287
1
vote
0
answers
50
views
Is there a name/reference for this dynamic process?
I'm just curious about the following dynamic process:
$$
x_{t+1} = x_t\frac{y_t + \varepsilon_t}{y_t + x_t}
$$
$$
y_{t+1} = y_t + \varepsilon_t
$$
Is there a name for this? I'm trying to find ...
- 149
0
votes
0
answers
11
views
Posterior Probabilities of Competing Univariate Linear Regression Models
Assume there are two potential univariate linear regression models $M1$ and $M2$, both of which have a prior probability of $0.5$.
Under $M1$, $Y=\alpha + \beta X_1 + \epsilon$, while under $M2$,$Y=\...
- 195
0
votes
0
answers
21
views
Exploring Likelihood Representation in Conditional Probability: Seeking Insights for Estimating $P[Y=y \mid X=x]$ in a Temperature Control System
Consider an information source which produces the signals 1 and 0 with unknown probabilities. The output of the source is denoted by the discrete variable Y. Y=1 implies that the temperature of a ...
- 1
2
votes
0
answers
42
views
How could I write the following sum in terms of expectation?
The following is from Bergemann and Morris (2016)
Suppose that $I$ is a finite set of players, where $i= 1,2,\dots, I$ and $i$ refers to the typical player. Let $\Theta$ be a finite set of states ...
- 115
0
votes
0
answers
54
views
Why does $Y'=\Theta+(W/3)$ have the same estimation and MSE as $Y=3\Theta+W$?
This is from MIT's 6.431x.
For the model $X=\Theta+W$, and under the usual independence and normality assumptions for $\Theta$ and $W$, the mean squared error (MSE) of the LMS estimator is
$$\frac{1}{...
- 1,438
0
votes
0
answers
62
views
german tank problem — frequentist answer derivation
I have a question on this famous problem called the German tank problem.
Assuming tanks are assigned sequential serial numbers starting with 1, suppose that four tanks are captured and that they have ...
- 85
0
votes
1
answer
143
views
german tank problem confusion — bayesian vs frequentist thinking
I'm having trouble understanding this famous problem known as the German tank problem. The problem goes as follows:
Assuming tanks are assigned sequential serial numbers starting with 1, suppose that ...
- 85
0
votes
0
answers
25
views
Likelihood calculation for Naive Bayes classifier
I am reading the Generative models for discrete data chapter in Kevin P Murphy's book(Machine Learning: A Probabilistic Perspective)
Here for calculating the MLE of naive Bayes (pg no: 83) the ...
0
votes
0
answers
47
views
Probability of a dice being fair/biased given certain rolls
I'm a philosophy student that has had to teach himself probabilities in order to get into formal epistemology. I have never taken a statistics/probabilities course, so I may be missing something very ...
2
votes
1
answer
84
views
Prove posterior mean is positive if and only if signal is positive, assuming zero prior mean [closed]
Suppose that:
$\Delta \sim G$, where $G$ is a distribution that is symmetric about the origin
I get a normally-distributed signal:
$\hat{\delta} \: |\Delta, \tau^2 \sim N(\Delta, \tau^2)$
How can I ...
- 29
0
votes
0
answers
22
views
How do you infer the model of a car based on prior information?
Sorry if this does not quite make sense as I am still wrapping my head around it as well. Suppose I have j car models (i.e. different brands, builds etc.) such that $\textbf{m} = {m_1, m_2, . . . , ...
- 21
0
votes
0
answers
30
views
Confusion with implementation of PDE constrained Bayesian Inverse Problem
Consider a PDE,
$$\partial_t u -a \nabla u - ru (1-u) = 0$$
at a given snapshot in time. The inverse problem is to find the diffusion coefficient $a \in L^{\infty}$ from a noisy measurement $$Y = \Phi(...
- 101