1
$\begingroup$

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other covariates in our regression model. Why would we include other covariates and how do you decide which of those 300 variables we want to include in our regression model?

$\endgroup$
1
$\begingroup$

There are many approaches to variable selection. First and foremost, you need to determine if each variable even makes sense for your question. For example, would you expect height to be related to the height of their in-laws? No...but it could be thrown into the variable mix anyway. First step is to use common sense to get an idea of what variables seem important, then you can get systematic Below are a couple major approaches:

-Stepwise regression (forward, backwards, etc)

-Lasso and ridge regression

$\endgroup$
1
$\begingroup$

Say you're trying to find the effect of dosage of a drug on some measure of symptoms. It might be that in reality, the drug works very differently in people depending on their gender, but if you account for the different genders, the effect of the drug can be more clear. This is just an oversimplified example of how including covariates can improve the ability to see a relationship.

To echo the above answer, stepwise regression is a great approach given that you have so many variables. I'm sure that your textbook has some info on this topic, or at least on the topic of comparing two nested models (meaning that one model is the same as the other model but with additional covariates). ANOVA and AIC are some common ways to compare nested models. As a rule of thumb, if two models are equally good, use the one with fewer variables to avoid overfitting.

Also, with 300 variables, multicollinearity (two variables explaining some of the same information) is a very real possibility, and some ways to check for it should also be described in your class resources.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.