0
$\begingroup$

Vectors $A, B$ and covariance matrix $C$ are fixed and known. I have a vector of measurements, $Y\in\mathbb{R}^n$, sampled from $$ M_1: Y \sim N(A\alpha_* + B\beta_*, C) $$

My goal, roughly speaking, is to find a small 50% Bayesian credible set, $S$, for $\alpha$ that is robust: $S$ should exhibit at least 50% coverage probability for all value of $\alpha$, meaning that if $Y\sim N(A\alpha_* + B\beta_*, C)$ for fixed $(\alpha_*, \beta_*)$, then $\alpha_* \in S$ with probability at least 50%.

The classical approach of jointly estimating $(\alpha, \beta)$ yields unacceptably large $S$ when $\beta=0$, since in this case we are estimating more degrees of freedom than are necessary. If we knew a priori that $\beta=0$, we would estimate compute $S$ by classical regression on $Y\sim N(A\alpha, C)$ and be done. Of course when $\beta \neq 0$, this approach results in degraded coverage probability for $S$.

I'm looking for a way to get smaller credible sets than classical methods afford when $\beta=0$ (or near zero) while maintaining coverage probability for all $\alpha_*, \beta_*$.

Here is one idea. Consider the Bayesian posterior distribution of $\alpha$ assuming flat (improper) prior distributions on $\alpha, \beta$: $$ P(\alpha | Y) \propto \int P(Y|\alpha, \beta)\, d\beta $$ where the likelihood function $P(Y|\alpha, \beta)$ is given by the normal distribution above. The posterior $P(\alpha|Y)$ is Gaussian with mean and variance determined by classical linear regression. Instead, introduce an informative prior $\pi(\beta)\propto P(Y|\beta)$, where $$ P(Y|\beta):=\int P(Y|\alpha, \beta)\, d\alpha $$ so that $$ P(\alpha|Y) \propto \int P(Y|\alpha, \beta) P(Y|\beta) \, d\beta $$ In cases were $\beta$ is near 0, the integrand should be more concentrated near the mode of the likelihood function, thus reducing the size of $S$. When $\beta \neq 0$ and is 'observable', $P(Y|\beta)$ should be concentrated near the correct value.

Using the same data in the likelihood and prior in this way feels abusive, but it's all I've come up with so far.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.