Consider the model $Y_i = \beta x_i + \varepsilon_i$ where $i = 1,\ldots, n$.

We know that $\varepsilon_1,\ldots, \varepsilon_n$ is iid sequence of random variables from $N(0,\sigma^2)$ and $x_i, i = 1,\ldots,n$ are given constants. Find MLE for both $\beta$ and $\sigma^2$. Find the distribution of $\hat \beta$. Is $\hat \beta$ an unbiased estimator?

So, basically, this is what I've done so far:

$$L(\beta,\sigma^2) = \left(\frac{1}{2\pi\sigma^2}\right)^{\frac{n}{2}} \exp\left[\frac{\sum(Y_i - \beta x_i)^2}{2\sigma^2} \right] \\ \frac{d(\ln L)}{d\beta} = 0 = \sum(Y_i - \beta x_i)(x_i) \implies \hat \beta = \sum \frac{Y_i}{x_i}$$

I'm not sure if I did this correctly, but if someone could find the errors, that would be helpful.

I also tried to find the MLE for $\sigma^2$:

$$\hat \sigma^2 = \frac {\sum (Y_i - \beta x_i)^2}{n}$$

I'm not quite sure if I'm doing any of these questions correctly, and I am not really sure how to figure out the distribution for $\hat \beta$ and whether it is an unbiased estimator or not. Like how can I determine the distribution of $\beta$ with the given information? Thanks!

Edit (To show more information):

$\sum x_i y_i - \beta \sum x_i^2 = 0 \implies \beta \sum x_i^2 = \sum x_i y_i \implies \beta = \frac{\sum x_i y_i}{\sum x_i^2} \implies \beta = \frac{y_i}{x_i}$.

  • $\begingroup$ MLE of $\sigma^2$ would depend on the MLE of $\beta$, so $\beta$ has to be replaced by $\hat\beta$ in $\hat\sigma^2$. Note that your very last equality in the edit is incorrect. $\endgroup$ – StubbornAtom Feb 9 '20 at 6:48

$$ \widehat\beta = \frac{\sum_{i=1}^n x_iY_i}{\sum_{i=1}^n x_i^2}. $$ The $Y$ in the expression above is capital; the $x$s are in lower case. That is to emphasize that $Y_i,$ in the problem as stated, is a random variable, and $x_i$ is not random. Therefore we have \begin{align} & \operatorname E\widehat\beta = \frac{\sum_{i=1}^n x_i \operatorname EY_i}{\sum_{i=1}^n x_i^2} = \frac{\sum_{i=1}^nx_i(\beta x_i)}{\sum_{i=1}^nx_i^2} = \beta. \\[12pt] & \operatorname{var}\widehat\beta = \frac{\sum_{i=1}^n x_i^2\operatorname{var}Y_i}{\left( \sum_{i=1}^n x_i^2 \right)^2} = \frac{\sum_{i=1}^n x_i^2 \sigma^2}{\left( \sum_{i=1}^n x_i^2\right)^2} = \frac{\sigma^2}{\sum_{i=1}^n x_i^2}. \end{align}

The only thing you need to know beyond that is that a linear combination of normally distributed random variables with constant (i.e. non-random) coefficients is normally distributed.


Dont have enough to comment, but you'll get the right answers if you are bit more careful. i.e. it should be $\Sigma (Y_i - \beta x_i)x_i$ that is, $x_i$ is within then sum so doesnt cancel out.

  • 1
    $\begingroup$ Hi, I added some more information, I'm not quite sure what you mean by saying it doesn't cancel out. Did i do something wrong up there^ $\endgroup$ – Bar Feb 9 '20 at 1:04
  • 2
    $\begingroup$ The second last equality is the right answer. The last step makes no sense. Maybe write out the sums of the numerator and denominator explicitly and you will see why. $\endgroup$ – Lulu Feb 9 '20 at 2:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.