# Why doen't we consider nonlinear estimators for the parameters of linear regression models?

The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the best linear unbiased estimator (BLUE) for the coefficients in a linear regression (given some conditions on the errors). I can understand why we want an unbiased and minimum-variance ("best") estimator, but why linear? Why not an estimator that depends on any other power (square, square root, etc) of the data?

More specifically, for an $n\times m$ data matrix $X$ predicting an $n \times 1$ response vector $y$ in the model $y = \beta X + \epsilon$, the OLS estimator for the coefficients $\beta$ is,

$$\hat\beta = (X^TX)^{-1}X^Ty = Cy = c_0 y_0 + c_1 y_1 + c_2 y_2 + \cdots$$

Note that $\hat\beta$ is defined linearly in terms of $y_i$ and thus a linear estimator. Is there a particular reason we don't consider estimators of the form, $$\tilde\beta = Cy^a = c_0 y_0^a + c_1 y_1^a + c_2 y_2^a + \cdots$$

• If $y$ is an $n \times 1$ response vector what would $y^a$ mean? – Henry Apr 28 '17 at 7:33
• Edited the original question, but $y^a$ means element-wise exponentiation. – user126350 Apr 28 '17 at 18:21

If you have a prior probability distribution of $\beta$ then it is reasonable to use the conditional expected value of $\beta$ given the data as an estimator of $\beta,$ and that is indeed sometimes done. That estimator is nonlinear. It is also biased, since it prefers more probable values of $\beta$ to less probable values.