Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually against $y$, all but $x_2$ come up significant (and four of those are $p < 0.001$). However, when all six are used in a multiple linear regression model, only two come up as significant.

This seems somewhat counter intuitive to me. I understand that multiple linear regression will look at how much each factor contributes to the overall model, but in the single regressions, it seemed like several variables were very important. Does it possibly have to do with scaling? In the single regressions, each variable produced a very different slope.

share|improve this question
add comment

1 Answer

up vote 3 down vote accepted

This makes total sense since your regressors aren't orthogonal to each other.

Refer to this wiki for a reference on collinearity:

http://en.wikipedia.org/wiki/Multicollinearity

share|improve this answer
    
OK, makes sense. I'm reading the "remedies" section, and I'm curious what the preferred solution is. It seems silly to not include certain values in my model, and the suggestion of "collect more data" is impossible in my case. –  allie Mar 10 '12 at 22:16
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.