# Bias and variance of IV estimation

NOTE: I have asked this question on stat.stackoverflow but got no answers/comments. Hence I decide to ask it on the math.stackoverflow platform as well.

I'm studying IV estimation by myself and have some confusion about the basics. Let $$y=X\beta_0 + u$$ be a linear model with endogenous variable $$X$$, and $$Z$$ be an instrument, meaning that $$Z$$ and $$u$$ are uncorrelated, and $$Z$$ and $$X$$ are correlated. For simplicity let's say both $$X$$ and $$Z$$ are univariate.

The IV estimation takes the form of $$\widehat\beta = \beta_0 + (Z'X)^{-1}Z'u.$$

I understand that $$\widehat\beta$$ is consistent because $$Z'u\overset{p}{\to} 0$$, $$Z'X\overset{p}{\to} a\neq 0$$ and therefore by Slutsky's theorem $$\widehat\beta\overset{p}{\to}\beta_0$$. However, I'm having some trouble on other properties of $$\widehat\beta$$.

1. Is $$\widehat\beta$$ unbiased? For me it seems $$\widehat\beta$$ is not unbiased, because $$Z'u$$ and $$Z'X$$ are correlated (are they?) and therefore $$\mathbb E[Z'u]=0$$ does not mean $$\mathbb E[(Z'X)^{-1}Z'u]=0$$?
2. If $$\widehat\beta$$ is indeed biased, how do we obtain the (asymptotic) variance of $$\widehat\beta$$? If we calculate by definition then $$\mathbb E|\widehat\beta-\beta_0|^2 = \mathbb E[(Z'X)^{-1}Z'uu'Z(Z'X)^{-1}]$$ but $$u$$ is not uncorrelated with $$(Z'X)^{-1}Z'$$, and how do we simplify this expression?

1) Yes, $$\hat{\beta}$$ is indeed biased:

$$E[\hat{\beta}] = \beta_{0} + E_{Z, X, u}[(Z'X)^{-1}Z'u] = \beta_{0} + E_{Z, X}[(Z'X)^{-1}Z'E(u\mid Z, X)]$$

via the law of iterated expectations. In order to have $$\hat{\beta}$$ unbiased we should have $$E(u\mid Z, X)$$ = $$0$$, but this is an assumption too strong since it would also imply $$E(u\mid X)$$ = $$0$$, case in which you would not even have an endogeneity problem with OLS estimators.

2) For the asymptotic variance of $$\hat{\beta}$$ consider that: $$\hat{\beta} = \beta_{0} + (Z'X)^{-1}Z'u = \beta_{0} + (n^{-1}Z'X)^{-1}n^{-1}Z'u$$

From which:

$$\sqrt{n} (\hat{\beta} - \beta_{0}) = \left ( \frac{1}{n}Z'X \right )^{-1}\frac{1}{\sqrt{n}}Z'u$$

and since we're assuming $$plim\left ( \frac{1}{n}Z'X \right )$$ $$\neq$$ $$0$$ and $$plim \left ( \frac{1}{n}Z'u \right )$$ = $$0$$, then the plim of left hand side converges to $$0$$.

In particular, define $$plim\left ( \frac{1}{n}Z'X \right )$$ = $$Q_{ZX}$$, $$plim\left ( \frac{1}{n}X'Z \right )$$ = $$Q_{XZ}$$ and $$plim\left ( \frac{1}{n}Z'Z \right )$$ = $$Q_{ZZ}$$.

At this point, I've worked out the solution as follows:

\begin{align} As.Var(\hat{\beta}) &= \frac{1}{n} plim E \left [ \left ( \frac{1}{n}Z'X \right ) ^{-1}\left (\frac{1}{\sqrt{n}}Z'u\right )\left (\frac{1}{\sqrt{n}}u'Z \right ) \left ( \frac{1}{n}X'Z \right ) ^{-1} \right ] \\ & = \frac{1}{n} E \left [ plim \left ( \frac{1}{n}Z'X \right ) ^{-1} plim \left (\frac{1}{\sqrt{n}}Z'u\right )plim \left (\frac{1}{\sqrt{n}}u'Z \right ) plim \left ( \frac{1}{n}X'Z \right ) ^{-1} \right ] \\ & = \frac{1}{n} E \left [ plim \left ( \frac{1}{n}Z'X \right ) ^{-1} plim \left (\frac{1}{n}Z'uu'Z \right ) plim \left ( \frac{1}{n}X'Z \right ) ^{-1} \right ] \\ &= \frac{1}{n} \left [plim \left ( \frac{1}{n}Z'X \right ) ^{-1} plim \left (\frac{1}{n}Z'E(uu')Z \right ) plim \left (\frac{1}{n}X'Z \right )^{-1} \right ] \\ &= \frac{1}{n} \sigma^{2}Q_{ZX}^{-1}Q_{ZZ}Q_{XZ}^{-1} \\ \end{align}

where we have used homoskedasticity, plim properties and Central Limit Theorem (explaining the $$\frac{1}{n}$$)

• Very clean explanations. Thank you so much! – Yining Wang Apr 20 at 8:38