Setting up the liklihood distribution for Bayesian Estimation

Question

Here is the exact problem:

Suppose that the random variables $Y_1,\ldots,Y_n$ satisfy $$ Y_i=\beta x_i+\varepsilon_i, \quad i=1,\ldots,n. $$ where $x_1,\ldots,x_n$ are fixed constants, and $\varepsilon_1,\ldots,\varepsilon_n$ are i.i.d random variables with $\mathrm{Normal}(0,\sigma^2)$. Given data observations, denoted as $D$, i.e., $D=\{x_i,y_i\}^n_{i=1}$, answer the following question through calculation.

Assume $\sigma^2$ is known and assume the prior distribution of $B$ follows normal distribution, i.e., $\beta\sim \mathrm{Normal}(B_0,\sigma_o^2)$, where $B_0$ and $\sigma_0^2$ are known constants. Determine the marginal posterior distribution of $B$, i.e. $B\mid D$.

Here is my work so far:

Thus, the pdf of the priori distribution is:

$$\pi(\beta)=\frac{1}{\sqrt{2\pi}\sigma_0}e^{\frac{-(\beta-\beta_0)^2}{2\sigma_0^2}}$$

Next, I need to find the distribution of the $Y_i$ random variable.

$$\operatorname{E}[Y_i]=\operatorname{E}[\beta x_i+\varepsilon_i]=\beta x_i+0=\beta x_i$$

$$\operatorname{Var}[Y_i]=\operatorname{Var}[\beta x_i+\varepsilon_i]=\sigma^2$$

Thus, $Y_i \sim \mathrm{Normal}(\beta x_i, \sigma^2)$

Now, here is where I need help.

I know a sufficient statistic for the mean of a normal distribution when the variance is known is the sample mean.

Thus, having $\overline{x}=\frac{1}{n}\sum^n_{i=1}(\beta x_i)$ the likelihood becomes:

$$f(\overline{x}\mid \beta x_i)=\frac{1}{\sqrt{2\pi}\sigma}e^{\frac{-(\overline{x}-\beta x_i)^2}{2\sigma^2}}$$

So, I am wondering If my set up of the likelihood is correct. I know the posterior distribution is typically represented by $f(\bf{x}\mid \theta) \propto f(\theta \mid \bf{x}) \cdot \pi(\theta)$. In this representation notice that the parameter $\theta$ is the same throughout, but in the way I set up the problem it is not due to $\beta x_i$ and $\beta$. So, I'm wondering if any one can shed some light on how I need to represent the liklihood.

Answer 1

Your proposed sufficient statistic assumes all $n$ of the normal distributions are the same, but they're not.

The conditional distribution of $Y_i$ given $B$ is $\mathrm{Normal}(Bx_i, \sigma^2 )$. The likelihood is $$ L(\beta) = \frac 1 {\sigma^{2n}} \exp\left( \frac{-1}{2\sigma^2} \sum_{i=1}^n ( y_i - \beta x_i )^2 \right). $$ This admits further simplification by doing some algebra, and that can identify a sufficient statistic.

The least-squares estimator of $\beta$ can be shown to be $$ \widehat\beta = \frac{\sum_i x_i y_i}{ \sum_i x_i^2}. $$ Then we have \begin{align} & \sum_{i=1}^n ( y_i - \beta x_i )^2 = \sum_i \Big( (y_i - \widehat\beta x_i) + (\widehat\beta x_i - \beta x_i)\Big)^2 \\[10pt] = {} & \left( \sum_i (y_i - \widehat\beta x_i)^2 \right) + \underbrace{2(\widehat \beta - \beta) \sum_i (y_i-\widehat\beta x_i) x_i}_\text{Show that this middle sum is $0$.} + \left(\widehat\beta-\beta\right)^2 \sum_i x_i^2. \end{align} You should ultimately end up with $\widehat\beta$ as the sufficient statistic.