Suppose we have some data points $\mathcal{D}=\{(x_i,y_i)\}_{0\le i\le N}$ and we would like to do some linear regression analysis on these points. Let's stick to two methods, least squares (i.e. minimising the L2 norm of the residues) and L1 regression (i.e. using L1 norm).
I have more often encountered the first rather than the latter. However the L1 method is much more rigid in the sense that it is less sensitive to outliers. So my question: L1 seems to be much better for linear regression, why would we care about L2? Some ideas/additional questions.
- L2 problems seem to be easier to solve than L1. In any case, L2 would probably ask less computer power (is this difference in calculations needed significant for large datasets $\mathcal{D}$?)
- Which method would be most used in an industry setting? Why? Or are these not used at all and does one use other methods?
- Is there a situation where L2 regression is preferred over L1 regression (for reasons relying on the practical meaning of the data or problem, not computational ones)?
