2
$\begingroup$

I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually against $y$, all but $x_2$ come up significant (and four of those are $p < 0.001$). However, when all six are used in a multiple linear regression model, only two come up as significant.

This seems somewhat counter intuitive to me. I understand that multiple linear regression will look at how much each factor contributes to the overall model, but in the single regressions, it seemed like several variables were very important. Does it possibly have to do with scaling? In the single regressions, each variable produced a very different slope.

$\endgroup$

1 Answer 1

4
$\begingroup$

This makes total sense since your regressors aren't orthogonal to each other.

Refer to this wiki for a reference on collinearity:

http://en.wikipedia.org/wiki/Multicollinearity

$\endgroup$
1
  • $\begingroup$ OK, makes sense. I'm reading the "remedies" section, and I'm curious what the preferred solution is. It seems silly to not include certain values in my model, and the suggestion of "collect more data" is impossible in my case. $\endgroup$ Commented Mar 10, 2012 at 22:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .