# How to compare asymptotic distribution of MLE and OLS estimators?

I have the following problem on my Statistics Problem Set:

Consider the model \begin{equation*} y_{i}=x_{i}^{\prime }\beta +u_{i}, \end{equation*} where $\left( x_{i}^{\prime },u_{i}\right)$ are iid with $u_{i}|x_{i}$ have the density $f\left( u\right) \in C^{2}$ (with support $-\infty <u<\infty$ ). Assume that \begin{eqnarray*} E\left( U\right) &=&\int_{-\infty }^{\infty }uf\left( u\right) =0\text{ and} \\ V\left( U\right) &=&E\left( U^{2}\right) =\int_{-\infty }^{\infty }u^{2}f\left( u\right) =\sigma ^{2}. \end{eqnarray*}

a)Use transformation of variables to show that the (conditional) pdf of $y_{i}|x_{i}$ is given by $g\left( y_{i}|x_{i}\right) =f\left( y_{i}-x_{i}^{\prime }\beta \right)$.

b) Find the likelihood of $y=\left( y_{1},...,y_{n}\right)$ conditional on $X=\left( x_{1},...,x_{n}\right) ^{\prime }$.

c) State the Gauss-Markov theorem.

d) We will show in item (f) that the asymptotic variance of$\sqrt{n}(% \widetilde{\beta }-\beta ^{\ast })$ can be smaller than the asymptotic variance of $\sqrt{n}(\widehat{\beta }-\beta ^{\ast })$, where $\widetilde{% \beta }$ is the MLE and $\widehat{\beta }$ is the OLS estimator. Explain why this result does not contradict the Gauss-Markov theorem.

e) Find the asymptotic variance of$\sqrt{n}(\widehat{\beta }-\beta ^{\ast })$.

f) Show algebraically that (i) the asymptotic variance of$\sqrt{n}(% \widetilde{\beta }-\beta ^{\ast })$ is no larger than the asymptotic variance of $\sqrt{n}(\widehat{\beta }-\beta ^{\ast })$; and (ii) give a necessary and sufficient condition on the density $f\left( u\right)$ for the asymptotic variance of $\widehat{\beta }$ and $\widetilde{\beta }$ to be the same.

I'm fine with this problem until item f).

I was able to do it all, including computing the asymptotic distributions:

$\sqrt{n}(\hat{\beta} - \beta ^{\ast }) \xrightarrow{d} N(0, \sigma^2(x^T x)^{-1})$

$\sqrt{n}(\tilde{\beta} - \beta ^{\ast }) \xrightarrow{d}N\Bigg(0, \Bigg[\mathbb{E}\Big(x_i \bigg(\frac{d \log(f(u))}{du}\bigg)^2 x_i'\Big)\Bigg]^{-1}\Bigg)$

Indeed, if $f(u)$ is Gaussian, then

$\mathbb{E}\Bigg(\Big(\frac{d\log(f(u))}{du}\Big)^2 \Bigg) = \mathbb{E}\Big(\frac{u^2}{\sigma^4} \Big) = 1/\sigma^2$,

which would take us to the previous case of OLS. I would feel happy answering at least the first part of item f), which asks us to compare these asymptotic variances. However, I don't see how to do it because we don't have so much structure on $f$. Any ideas on how to it? Thanks in advance!

• Actually the asymptotic distributions you wrote are all incorrect. I think you meant $$\sqrt{n}(\hat{\beta} - \beta ^{\ast }) \xrightarrow{d} N(0, \sigma^2(x^T x)^{-1})$$ – Ahmad Bazzi Sep 7 '18 at 12:23
• @AhmadBazzi Well, if $f$ is a Gaussian pdf, then $\log f(u) = c - \frac{u^2}{2\sigma^2}$. Then the result follows, doesn't it? – Raul Guarini Sep 7 '18 at 12:25
• @AhmadBazzi but how come the distribution has a 1x1 asymptotic variance? Is your $x$ a matrix? – Raul Guarini Sep 7 '18 at 12:27
• All i'm saying is if $x = [x_1 \ldots x_n]$, i.e. your observed samples, then the asymptotic distribution on the OLS estimator (given zero mean and iid $u_i$'s of course) is as I mentioned above. What you wrote doesn't make sense because according to your writing, you say that the asymptotic distribution on the OLS estimator depends on $x_i$ !!!! which $x_i$ ? $x_1$ ? $x_n$ ? any ? – Ahmad Bazzi Sep 7 '18 at 14:28
• @AhmadBazzi yes, since they are iid it makes no difference! There is a slight confusion on notation however, I believe. The full model I have in mind is $y = X\beta + u$ where $X$ is a $n \times k$ matrix and $\beta$ is $k \times 1$. So, $X'u$, for example, can be seen as $\sum_{i = 1}^{n}u_i x_i$, where $x_i$ is a $k \times 1$ vector. Does it make sense now? – Raul Guarini Sep 7 '18 at 14:40