0
$\begingroup$

Say I have a years worth of data on donut sales, and I use linear regression with variables such as daily occupancy, daily weather etc. because there is a good correlation between each variable and donut sales. As I am developing this model I find better variables and get a more accurate representation, which is noted by the increased adjusted R-squared value.

Would me improving this model be the best way to improve the forecast for projected sales? I want to very accurately predict my sales over the next year.

I thought the method was a good way to forecast, but I am starting to have doubts because although I am putting much time trying to essentially improve the fit of old data, since forecasting is more random am I just wasting time?

Are there any tips for better forecasting techniques other than linear regression? Thanks

$\endgroup$
  • $\begingroup$ If you use the wheather as input, wouldn'T a forecast over next year require you to make a wheather forecast over next year to begin with? $\endgroup$ – Hagen von Eitzen May 29 '14 at 15:29
  • $\begingroup$ Letting too many variables play a role may make the prediction essentially worthless. I always snicker when hearing things like that someone has a 67% success average in the running season when hitting against a lefthanded opponent in rainy weather, as it probably just means that this exact convoluted situation has not occured very often yet. $\endgroup$ – Hagen von Eitzen May 29 '14 at 15:36
  • $\begingroup$ Not necessarily. If we believe the adjusted R^2 is a good penalty against overfitting, then chances are that a good model for the past should be ok for the future. If we can't use past knowledge what can we use? $\endgroup$ – PA6OTA May 29 '14 at 15:56
2
$\begingroup$

Improving a model's accuracy does not necessarily improve the model's prediction.

Over-fitting a model is one of the biggest mistakes many people make when creating models and it is also one of the hardest things to avoid.

As an example if we wanted to do a linear regression on 10 data points we could take 10 explanatory variables and get an $R^2$ value of 1. However it is also possible that we could explain 99% of the data using only 9 explanatory variables.

There are multiple ways to avoid over-fitting your data. You can and should use some of these techniques in linear regression; cross-validation and regularization.

There are also other techniques such as PCA if you are worried about multicolinearity.

Ridge regression also helps in these cases and is very useful for predictive models but no so much for inference.

In any case you can use Akaike information criterion and Bayesian information criterion to help you determine an appropriate number of explanatory variables.

$\endgroup$
  • $\begingroup$ +1 for mentioning that omnipresent term in modelling "overfitting". $\endgroup$ – Chinny84 May 29 '14 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.