We have $$\mathbb{E}[Y_i| X_i] = β_0 + β_1X_i$$ with the following characteristics. The conditional distribution of $Y_i$ given $X_i$ normal, as follows. $$Y_i| X_i ∼ N(β_0 + β_1X_i, σ^2)$$ The distribution of $X_i$ is fixed in repeated samples.

The error term $ε_i$ is: $ε_i = Y_i − β_0 − β_1X$.

Under the hypotheses above, $ε_i$ follows an unconditional i.i.d. normal distribution. $$ε_i ∼ N(0,σ^2)$$

This model has 3 parameters : $θ = (β_0, β_1, σ^2)$.

What would be the Method of Moments estimator and MLE for this model? Construct the information matrix for $θ = (β_0, β_1, σ^2)$

Here is what I've done so far: Starting with the MM:

The three population moments are as follows:

1- $\mathbb{E}[y_1-\beta_0-\beta_1x_i]=0$

2- $\mathbb{E}[(y_1-\beta_0-\beta_1x_i)x_i]=0$

3- $\mathbb{E}[ε_i^2]=\sigma^2$

So I generated the sample analogs to the three populations moment conditions:

1- $\frac{1}{N}\sum_{i=1}^{n}(y_1-\hat{\beta_0}-\hat{\beta_1}x_i)=0$


3- $\frac{1}{N}\sum_{i=1}^{n}\hat{ε_i^2}=\hat{\sigma^2}$

Now moving on to the MLE:

Based on the $pdf$ of the normal distribution: $$f_{Y_i}(y; β_0, β_1, σ^2)= \frac{1}{\sqrt{ 2πσ^2}} e^{{-\frac{1}{2σ^2}}(y-\beta_0-\beta_1x_i)^2}$$

The log likelihood is: $$log(L(β_0, β_1, σ^2)= -\frac{n}{2}log(σ^2)−\frac{1}{2σ^2}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)^2$$

and have found the 3 first derivatives of $\beta_0,\beta_1$ and $\sigma^2$ for the MLE

$\frac{\partial log(L(β_0, β_1, σ^2)}{\partial \beta_0}= \frac{1}{\sigma^2}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)=0$

$\frac{\partial log(L(β_0, β_1, σ^2)}{\partial \beta_1}= \frac{1}{\sigma^2}\sum_{i=1}^{n}x_i(y_1-\beta_0-\beta_1x_i)=0$

$\frac{\partial log(L(β_0, β_1, σ^2)}{\partial \sigma^2}= -\frac{n}{2\sigma^2}+\frac{1}{2(\sigma)^{4}}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)^2=0$

Multiplying throughout by $\frac{2\sigma^4}{N}$ and rearranging the result, gives $$\sigma^2(\beta_0,\beta_1)=\frac{1}{N}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)^2=\frac{1}{N}\sum{e_i^2}$$

I have no idea about the information matrix.


1 Answer 1


The observed Fisher information matrix is $$ I(\hat{\theta}) = - \frac{\partial ^ 2}{\partial \theta_i \partial \theta_j} \big|_{\theta = \hat{\theta}}. $$ For $I(\hat{\theta})_{11} = - \frac{\partial}{\partial \beta_0} \sigma^{-2}\sum (y_i - \beta_0 - \beta_1x_i) = n/\sigma^2\Big|_{\sigma = \hat{\sigma}} = n/\hat{\sigma}^2$.

For $I(\hat{\theta})_{12} = I(\hat{\theta})_{21} = - \frac{\partial}{\partial \beta_1} \sigma^{-2}\sum (y_i - \beta_0 - \beta_1x_i) = n\bar{x}_n/\sigma^2\Big|_{\sigma = \hat{\sigma}} = n\bar{x}_n/\hat{\sigma}^2$.

For $I(\hat{\theta})_{22} = - \frac{\partial}{\partial \beta_1} \sigma^{-2}\sum (y_i - \beta_0 - \beta_1x_i)x_i = \sum x_i^2/\sigma^2\Big|_{\sigma = \hat{\sigma}} = \sum x_i^2/\hat{\sigma}^2$.

For $I(\hat{\theta})_{33} = - \frac{\partial}{\partial \sigma^2} \left( \frac{\partial}{\partial \sigma^2} \ln L(\theta;x_1,...,x_n) \right)=...= \frac{n}{2\hat{\sigma}^4} $.

$I(\hat{\theta})_{13} = I(\hat{\theta})_{23} = 0$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .