0
$\begingroup$

I have an, I think, rather basic question about linear regression to which I can not find a satisfactory answer.

I am trying to apply linear regression on a certain data set. First of all, I have detected outliers and high leverage values using studentized deleted residuals and leverage values. For this, I have just applied the most basic linear regression model on the data ($y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + ...$). I have done nothing with these outliers, didn't remove them from the data set.

After this, I have taken a closer look at the data and found out that I should transform the dependent variable y to ln(y). Furthermore, I have removed a couple of redundant variables to obtain a final model.

However, when I try to detect outliers and high leverage values using studentized deleted residuals and leverage values again, but this time with my final model, I obtain different outliers!

Now my question is: should you first detect outliers and then build a model, or the other way around?

Thanks very much for your help in advance!

$\endgroup$
0
$\begingroup$

Outliers can result from an inappropriate model. E.g., assume that the real model that generates that data is $y=bx^2 + \epsilon$, however you have fitted $y=b_0 + b_1x$, in such case you may observe huge amount of "outliers" only because you misspecified the model. Thus, you should view the outliers also as an indication of a bad fitting, as such you should try to fit the best model first and only then to take care of the outliers. Eventhough, deleting outliers is pretty bad practice as you don't know what caused them, hence by such practice you may overlook some valuable information.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.