# Method of moment, MLE and information matrix

We have $$\mathbb{E}[Y_i| X_i] = β_0 + β_1X_i$$ with the following characteristics. The conditional distribution of $$Y_i$$ given $$X_i$$ normal, as follows. $$Y_i| X_i ∼ N(β_0 + β_1X_i, σ^2)$$ The distribution of $$X_i$$ is fixed in repeated samples.

The error term $$ε_i$$ is: $$ε_i = Y_i − β_0 − β_1X$$.

Under the hypotheses above, $$ε_i$$ follows an unconditional i.i.d. normal distribution. $$ε_i ∼ N(0,σ^2)$$

This model has 3 parameters : $$θ = (β_0, β_1, σ^2)$$.

What would be the Method of Moments estimator and MLE for this model? Construct the information matrix for $$θ = (β_0, β_1, σ^2)$$

Here is what I've done so far: Starting with the MM:

The three population moments are as follows:

1- $$\mathbb{E}[y_1-\beta_0-\beta_1x_i]=0$$

2- $$\mathbb{E}[(y_1-\beta_0-\beta_1x_i)x_i]=0$$

3- $$\mathbb{E}[ε_i^2]=\sigma^2$$

So I generated the sample analogs to the three populations moment conditions:

1- $$\frac{1}{N}\sum_{i=1}^{n}(y_1-\hat{\beta_0}-\hat{\beta_1}x_i)=0$$

2-$$\frac{1}{N}\sum_{i=1}^{n}x_i(y_1-\hat{\beta_0}-\hat{\beta_1}x_i)=0$$

3- $$\frac{1}{N}\sum_{i=1}^{n}\hat{ε_i^2}=\hat{\sigma^2}$$

Now moving on to the MLE:

Based on the $$pdf$$ of the normal distribution: $$f_{Y_i}(y; β_0, β_1, σ^2)= \frac{1}{\sqrt{ 2πσ^2}} e^{{-\frac{1}{2σ^2}}(y-\beta_0-\beta_1x_i)^2}$$

The log likelihood is: $$log(L(β_0, β_1, σ^2)= -\frac{n}{2}log(σ^2)−\frac{1}{2σ^2}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)^2$$

and have found the 3 first derivatives of $$\beta_0,\beta_1$$ and $$\sigma^2$$ for the MLE

$$\frac{\partial log(L(β_0, β_1, σ^2)}{\partial \beta_0}= \frac{1}{\sigma^2}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)=0$$

$$\frac{\partial log(L(β_0, β_1, σ^2)}{\partial \beta_1}= \frac{1}{\sigma^2}\sum_{i=1}^{n}x_i(y_1-\beta_0-\beta_1x_i)=0$$

$$\frac{\partial log(L(β_0, β_1, σ^2)}{\partial \sigma^2}= -\frac{n}{2\sigma^2}+\frac{1}{2(\sigma)^{4}}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)^2=0$$

Multiplying throughout by $$\frac{2\sigma^4}{N}$$ and rearranging the result, gives $$\sigma^2(\beta_0,\beta_1)=\frac{1}{N}\sum_{i=1}^{n}(Y_i − β_0 − β_1x_i)^2=\frac{1}{N}\sum{e_i^2}$$

I have no idea about the information matrix.

The observed Fisher information matrix is $$I(\hat{\theta}) = - \frac{\partial ^ 2}{\partial \theta_i \partial \theta_j} \big|_{\theta = \hat{\theta}}.$$ For $$I(\hat{\theta})_{11} = - \frac{\partial}{\partial \beta_0} \sigma^{-2}\sum (y_i - \beta_0 - \beta_1x_i) = n/\sigma^2\Big|_{\sigma = \hat{\sigma}} = n/\hat{\sigma}^2$$.

For $$I(\hat{\theta})_{12} = I(\hat{\theta})_{21} = - \frac{\partial}{\partial \beta_1} \sigma^{-2}\sum (y_i - \beta_0 - \beta_1x_i) = n\bar{x}_n/\sigma^2\Big|_{\sigma = \hat{\sigma}} = n\bar{x}_n/\hat{\sigma}^2$$.

For $$I(\hat{\theta})_{22} = - \frac{\partial}{\partial \beta_1} \sigma^{-2}\sum (y_i - \beta_0 - \beta_1x_i)x_i = \sum x_i^2/\sigma^2\Big|_{\sigma = \hat{\sigma}} = \sum x_i^2/\hat{\sigma}^2$$.

For $$I(\hat{\theta})_{33} = - \frac{\partial}{\partial \sigma^2} \left( \frac{\partial}{\partial \sigma^2} \ln L(\theta;x_1,...,x_n) \right)=...= \frac{n}{2\hat{\sigma}^4}$$.

$$I(\hat{\theta})_{13} = I(\hat{\theta})_{23} = 0$$.