Given this regression model: $y_{i}=\beta_{0}+\beta_{1}x_{i}+E_{i}$.

All the assumptions are valid except that now: $E_{i}\sim N(0,x_{i}\sigma^{2})$

Find Maximum likelihood parameters for $\beta_0$,$\beta_1$

  • $\begingroup$ Some context is missing. You say "this regression model". The model must have been specified before that, but you haven't told us about that. $\endgroup$ – Michael Hardy Nov 18 '15 at 5:51
  • $\begingroup$ This is the model $$y_{i}=\beta_{0}+\beta_{1]x_{i}+E_{i}$$ as writing $\endgroup$ – Dkova Nov 18 '15 at 7:21
  • $\begingroup$ If I'm reading this right, what is unique about this problem is that the variance is proportional to $x_i$. That implies that $x_i\geq 0$, necessarily. But can we go further and assume that $x_i>0$ in all cases? $\endgroup$ – Michael Grant Nov 18 '15 at 15:05

What makes the model interesting is that the variance is proportional to the data $x_i$. This implies that $x_i\geq 0$. I am going to start by assuming that $x_i>0$. But once we've found the answer in that case I'll discuss the situation when that assumption does not hold; i.e., that $x_i=0$ for at least one $i$.

Let us define the following quantities: $$\bar{y}_i = x_i^{-1/2} y_i, \quad \bar{x}_{i-} = x_i^{-1/2}, \quad \bar{x}_{i+} = x_i^{+1/2}, \quad \bar{E}_i = x_i^{-1/2} E_i$$ Then the equation becomes $$ \bar{y}_i = \beta_0 \bar{x}_{i-} + \beta_1 \bar{x}_{i+} + \bar{E}_i, \quad \bar{E}_i \in \mathcal{N}(0,\sigma^2)$$ What we have now is a standard, unweighted linear regression in two variables.

To solve this we define two matrices $$X = \begin{bmatrix} \bar{x}_{1-} & \bar{x}_{1+} \\ \vdots & \vdots \\ \bar{x}_{n-} & \bar{x}_{n+} \end{bmatrix}, \quad Y = \begin{bmatrix} \bar{y}_1 \\ \vdots \\ \bar{y}_n \end{bmatrix}$$ Then we have $\beta=(\beta_0,\beta_1)=(X^TX)^{-1}X^TY.$ But the special structure of $X$ and $Y$ leads to some interesting looking structure when we multiply it out: \begin{equation} X^TX = \begin{bmatrix} \sum_i x_i^{-1} & n \\ n & \sum_i x_i \end{bmatrix}, \quad X^T Y = \begin{bmatrix} \sum_i x_i^{-1} y_i \\ \sum_i y_i \end{bmatrix} \qquad \beta = \begin{bmatrix} \beta_0 \\ \beta_1 \end{bmatrix} = \frac{1}{ n^2 - \sum_i x_i^{-1} \sum_i x_i} \begin{bmatrix} n \sum_i y_i - (\sum_i x_i) \sum_i x_i^{-1} y_i \\ n \sum_i x_i^{-1} y_i - (\sum_i x_i^{-1}) \sum_i y_i \end{bmatrix} \end{equation}

What happens if we cannot assume $x_i>0$? Well, in that case the original relation reduces to $y_i=\beta_0$ and $E_i=0$. So that means that $\beta_0$ is forced to be equal to $y_i$ in that case. If there are multiple values of $x_i>0$, then we have two scenarios:

  • All of the corresponding values of $y_i$ are identical, in which case $\beta_0=y_i$.
  • The corresponding values of $y_i$ are not identical, in which case the model is infeasible and cannot be solved.

If the model is feasible, then we fix $\beta_0$ and eliminate those $(x_i,y_i)$ pairs from the model, leaving us with a single-variable regression. I hope you don't mind if I skip right to the result: $$\bar{z}_i = \beta_1 \bar{x}_{i+} + \bar{E}_i, \quad \bar{z}_i \triangleq \bar{y}_i - \beta_0 \bar{x}_{i-}\quad\Longrightarrow\quad \beta_1 = \frac{(\sum_i y_i) - n\beta_0}{\sum_i x_i}.$$

  • $\begingroup$ Are you sure that $y_i=\beta_0$ $\endgroup$ – Dkova Nov 22 '15 at 5:03
  • $\begingroup$ Only if one of the X values is zero $\endgroup$ – Michael Grant Nov 22 '15 at 12:18
  • $\begingroup$ I didn't understand, why not just the Maximum Likelihood Estimation? $\endgroup$ – Dkova Nov 23 '15 at 9:45
  • $\begingroup$ That is what this is, but the zero case requires extra care. $\endgroup$ – Michael Grant Nov 23 '15 at 12:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.