Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually against $y$, all but $x_2$ come up significant (and four of those are $p < 0.001$). However, when all six are used in a multiple linear regression model, only two come up as significant.

This seems somewhat counter intuitive to me. I understand that multiple linear regression will look at how much each factor contributes to the overall model, but in the single regressions, it seemed like several variables were very important. Does it possibly have to do with scaling? In the single regressions, each variable produced a very different slope.

share|cite|improve this question
up vote 2 down vote accepted

This makes total sense since your regressors aren't orthogonal to each other.

Refer to this wiki for a reference on collinearity:

share|cite|improve this answer
OK, makes sense. I'm reading the "remedies" section, and I'm curious what the preferred solution is. It seems silly to not include certain values in my model, and the suggestion of "collect more data" is impossible in my case. – allie Mar 10 '12 at 22:16

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.