# What is the Least Squares Estimator

A simple question.

When people talk about "the least square estimator", what is this estimator?

Is it an unbiased estimator of the slope of the regression line?

In a paper I'm reading, Let's Take the Con Out of Econometrics, the author writes

Randomization implies that the least squares estimator is "unbiased," but that definitely does not mean that for each sample the estimate is correct. (pg 31, last par)

I understand the second half of the sentence, but I don't understand why "randomization implies that the least squares estimator is 'unbiased.'" I do know that an estimator is unbiased if the expectation of the estimator is equal to the true value of the parameter.

Note: I get the impression that cross validated is for more professional statisticians, not for undergraduate level questions, hence I am posting on this site where I have seen more undergraduate level questions. Is that accurate?

• I think its fine to ask here, but you can also ask on Cross Validated. I've seen a lot of basic questions there and you'll get a lot of people for whom statistics is their thing. – roundsquare Jan 14 at 15:52

In the usual situation you have a model of the form $$Y=X\beta+\varepsilon$$ where $$X$$ is a $$n \times k$$ matrix (where $$n$$ is the number of samples), $$\beta$$ is a $$k \times 1$$ column vector of unknown coefficients (where $$k$$ is the number of coefficients in the model), and $$\varepsilon$$ is a $$n \times 1$$ column vector of noises. In this case the ordinary least squares estimator $$\hat{\beta}$$ is the minimizer of $$\| Y-X\hat{\beta} \|^2$$. When $$X$$ is a full rank matrix (which is typical if $$k and the variables in the model are independent), this minimizer is unique, and is given by $$(X^T X)^{-1} X^T Y$$. (Note that this is a bad algorithm to use numerically, there are better algorithms for solving a least squares problem numerically.)