Consider two simple linear models.

$y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and
$y_{2j}=\alpha _2+\beta_{2}x_{2j}+\epsilon_{2j}$ , $ j=1,2,\ldots,n>2$ where $ \epsilon_{ij} \sim N(0,\sigma^2)$
I have two questions.

1 ) Show that the point on the $x$-axis where the two lines intersect is given by $x_0={\alpha_1-\alpha_2 \over \beta_2-\beta_1}$
2)Obtain the maximum likelihood estimators of $\sigma^2$ and $x_0$

For question 1) what I did was,
Supposing at $x_0,y_0$ the two lines intersect I substituted $x_0,y_0$ to the two equations.But then what happens to the error terms.Can I suppose that the errors in estimating $x_0,y_0$ from both equations are the same?

For 2)

$y_{ij}$ has a Normal distribution.
I calculated $E(y_{1,j})$ and $E(y_{2,j})$ separately.Can I say that $E(X_{1j}) = E(X_{2j})$
and $E(y_{1,j}) = E(y_{2,j})$.
Thereby I obtained $E(y_{ij})={\beta_2\alpha_1-\beta_1\alpha_2 \over\beta_2-\beta_1}$

Similarly $V(y_{ij})=\sigma^2$

Then joint probability density function for $y_{ij}$ is $\prod_{i=1}^2\prod_{j=1}^n {1\over\sqrt {2\pi\sigma^2}}$$ e^{{-(y_{ij}-\mu})^2} / 2\sigma^2$ where $\mu= {\beta_2\alpha_1-\beta_1\alpha_2\over\beta_2-\beta_1}$. Is this a correct joint p.d.f for $y$?

Does $x_0$ also has a normal distribution. Is $V(x_0)=0$. I don't understand how to obtain MLE for $x_0$ .

Can someone please help me to do this please

  • $\begingroup$ The model uses the terms $\beta_{ij}$. In that case what do you mean by $\beta_1$ and $\beta_2$? $\endgroup$ – ved Jul 7 '14 at 4:07
  • $\begingroup$ @ved : Sorry that was a mistake.I corrected it $\endgroup$ – sam_rox Jul 7 '14 at 5:57
  • $\begingroup$ Assume x's are constants. $\endgroup$ – ved Jul 7 '14 at 13:01
  1. suppose the errors are zero to find the "ground truth" intersection point.
  2. In a vector-matrix notation this problem can be augmented to reflect all measurements: ${\bf{y}}={\bf{H}} \boldsymbol{\theta}+\boldsymbol{\epsilon}$, where ${\bf{y}}$ is a $2n \times 1$ column vector of all $y$'s concatenated,$\boldsymbol{\theta}=(\alpha_1,\beta_1,\alpha_2,\beta_2)^T$ and ${\bf{H}}$ is a $2n \times 4$ block matrix given by

\begin{equation} {\bf{H}} = \begin{pmatrix} {\bf{H_1}} & {\bf{0_{n \times 2}}}\\ {\bf{0_{n \times 2}}} & {\bf{H_2}} \end{pmatrix} \end{equation}

where ${\bf{H_1}}$ and ${\bf{H_2}}$ are $n \times 2$ Vandermonde matrices and are given by \begin{equation} {\bf{H_1}}=\begin{pmatrix} {\bf{1}}_{n \times 1} & {\bf{x_1}} \end{pmatrix} \end{equation}

\begin{equation} {\bf{H_2}}=\begin{pmatrix} {\bf{1}}_{n \times 1} & {\bf{x_2}} \end{pmatrix} \end{equation}

and the augmented error vector is of size $2n \times 1$ ,$\boldsymbol{\epsilon} \sim {\mathcal{N}}({\bf{0}}, \sigma^2 {\bf{I}}_{2n})$.

Thus, the Maximum Likelihood (ML) estimator for $\theta$ for this problem is given by the least squares (LS) estimator and is

\begin{equation} \widehat{\boldsymbol{\theta}}_{ML} = ({\bf{H}}^T{\bf{H}})^{-1} {\bf{H}}^T {\bf{y}} \end{equation}

so we get the ML estimation for the four parameters $\widehat{\boldsymbol{\theta}}_{ML}=(\widehat{\alpha}_{1_{ML}},\widehat{\beta}_{1_{ML}},\widehat{\alpha}_{2_{ML}},\widehat{\beta}_{2_{ML}})^T$ and by the invariance property of the ML estimator we get the ML estimator of $x_0$:

\begin{equation} {\widehat{x}}_{0_{ML}} = \frac{\widehat{\alpha}_{1_{ML}} - \widehat{\alpha}_{2_{ML}}}{\widehat{\beta}_{2_{ML}} - \widehat{\beta}_{1_{ML}}} \end{equation}

To complete the question the sample covariance $\sigma^2$ is derived by differentiating the joint log likelihood PDF and is given by

\begin{equation} {\widehat{\sigma}}^2_{ML} = \frac{1}{N} \left \| {\bf{y}} - {\bf{H}} \boldsymbol{\widehat{\theta}}_{ML} \right \|^2_2 \end{equation}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.