GMM = generalized method of moments

IV = instrumental variables

2SLS = Two stage least squares

OLS = ordinary least squares

I keep seeing talk of 'moment conditions' or 'moment equations', but don't exactly understand the context.

Consider a very standard regression model: $$y_i = \beta x_i + u_i $$

where $u_i$ is an error term, and suppose all the classic linear regression assumptions hold.

If I relax the exogeneity assumption,i.e., $\mathbb{E}(u|x) \neq 0$ (also side question: Why does this imply that $\mathbb{E}(u_i x_i)\neq0$?), then using OLS here will produce biased estimates right?

Is $\mathbb{E}(u_i | x_i)=0$ the 'moment condition' in OLS? Is it $\mathbb{E}(u_i x_i) =0$ ?

My second question is whether GMM, 2SLS, and IV are specifically distinct from one another.

My book says that when we have $K$ endogeneous regressors and $K$ instruments (exactly identified) we use IV.

In the case of being over-identified, and we have $J>K$ IVs, we use GMM. What about for the under-identified case?

Finally, What's the best way to distinguish between these different methods? For instance, what is the difference in using GMM in an over-identified case vs trying to use IV in that case?

Thanks for any help.

  • 1
    $\begingroup$ Sorry - updated it now. Thanks $\endgroup$ – codenoob May 11 at 21:31

The moment condition is $$ \mathbb{E}[u_i x_i] = 0 = \mathbb{E}[(y_i - \beta x_i)x_i]. $$ This is also called an orthogonality condition. It is used as a method of moments estimator by using the sample version, $$ (y-X \beta)X = 0, $$ and you solve for $\beta$ to get $$ \hat{\beta} = (X'X)^{-1} X'y, $$ the same as the OLS estimator.

Second question: GMM is the generalized method of moments. The standard method of moments is to pick sample moments that converge in probability to the true moment. For example, the sample mean is a method of moments estimator of the population mean by a weak law of large numbers. In GMM, the model can be non-linear, so that you have a population model $$ \mathbb{E}[g(X,\mu)] =0 $$ and are trying to pick $\mu$ to get the sample version to fit: $$ \dfrac{1}{N}\sum_{i=1}^N g(x_i,\mu) =0. $$

Instrumental variables is a tool for solving endogeneity or omitted variables bias or measurement error problems. In the linear framework, you have \begin{eqnarray} x = \pi' Z + \eta \\ y = \beta' x + \varepsilon \end{eqnarray} with $cov(\eta, \varepsilon) \neq 0$, OLS is biased. It is an independent concept of GMM, although you can use a GMM-IV framework to estimate models, which is nice because it gives a J-test of overidentification (which shows up later in your question). IV uses a different orthogonality condition, $$ (y-\beta X) Z =0 $$ where $Z$ are the instruments, and the IV estimator is $$ \hat{\beta} = (X'Z)^{-1}Z'y. $$ 2SLS is a way of getting the IV estimator with the least pain possible. You run OLS of $X$ on $Z$ to get $\hat{X}$, then regress $y$ on $\hat{X}$ instead of $X$ to get an unbiased estimate of $\hat{\beta}$. It is not GMM, just another way to do IV, although they are asymptotically (maybe even numerically?) equivalent.

If your model is under-identified, you don't have enough variables to solve the orthogonality condition. There are an infinite number of possible $\hat{\beta}$'s that solve whatever you are looking at. Don't do this, or look into the LASSO.

There is a category error in your last question. GMM is a method of finding and proving properties of estimators, like maximum likelihood. IV is a framework for estimating systems that violate the independence assumptions of OLS. The tests come out of the frameworks.

It sounds like you are reading Hayashi's book before you are ready. Try Greene or Kennedy, or Mostly Harmless Econometrics to get a background on this stuff.

| cite | improve this answer | |
  • $\begingroup$ Thank you, this is incredibly helpful. $\endgroup$ – codenoob May 11 at 22:08
  • $\begingroup$ So I just spent a few hours reading up on this, and much of it is clearer now, and I really appreciate your summary. I just had a quick question, when calculating the IV you said 2SLS is the least painful way possible. Assuming we have the same number of IVs and Endogenous variables, can we not just directly solve for the IV estimator b=(X'Z)^(-1)Z'y ? $\endgroup$ – codenoob May 12 at 2:33
  • 1
    $\begingroup$ Sorry, I guess I meant that 2SLS is conceptually less painful than the matrix inversion. 2SLS is, "project $X$ on $Z$, then $\hat{X}$ on $y$", and you can illustrate it easily to students using canned OLS commands, rather than a black box IV command. On the other hand, the IV normal equations are $(X'Z)\beta = Z'y$, and this illustrates why the first stage is important: $X'Z$ must be invertible. But in practice, we just look at the F stat of the first stage of 2SLS to check for weak instruments, since IV is not unbiased. I think it's safe to say that most applied people think of IV as 2SLS. $\endgroup$ – user762914 May 12 at 3:26
  • 1
    $\begingroup$ If I were you, and you are learning this stuff for a professional purpose and not just as a student, I would download R and try to replicate everything from scratch and monte carlo the asymptotic results to really understand what is going on. It is much easier to learn the material that way, compared to just reading endless pages of symbols and meditating on what they mean. Especially in stats/metrics, which is an inherently "imprecise" field, since most results only hold asymptotically under the null hypothesis. It is easy to misunderstand what a result says, or what its limits are. $\endgroup$ – user762914 May 12 at 3:29
  • 1
    $\begingroup$ Oh, the other thing to keep in mind is that the standard errors for IV require thought. If you run OLS twice, the second stage regressors will have standard errors that fail to account for the endogenous regressor. Using the GMM-IV framework you can get SE's, or use asymptotic formulas, or bootstrap them if you are so inclined. $\endgroup$ – user762914 May 12 at 3:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.