0
$\begingroup$

Let me illustrate the question with an example: Imagine you construct a model to predict tomorrow's rainfall over London using some variables. Let's suppose that 300 years of data are available (1718-2018). Of those aprox. 110.000 points, you use 70.000 to train you model and 40.000 as a test set. If 95 % of the residuals of the test set range between -5 +5 mm:

1.Would it be correct to say that the uncertainty of the predicted values for 2019 would be +-5 mm or whatever asymmetric interval we get?

2.Does the answer to question no 1 depend in any sense on the theoretical distribution of the residuals according to the structure and/or assumptions of the model? (remember that the sample is large and representative).

$\endgroup$
0
$\begingroup$

1: No, it is not correct !

Whatever you do, the prediction band will get larger and larger with time. Even if the error was systematically less than 5mm during the previous 300 years, there is no guarantee it will still be the same in the next 30'000 years.

enter image description here

  1. No, the answer to (1) is negative for any distribution.

Furthermore, if you want to estimate the widening of the confidence band, you must know the underlying distribution.

$\endgroup$
  • $\begingroup$ The prediction band will get larger and larger with time, but I proposed a 1 day forecast model. When I said *the predicted values for 2019 * I was talking about making predictions 1 day ahead within the 2019 year. $\endgroup$ – Alf Jun 20 at 18:21
0
$\begingroup$

I have just noticed that, for a 1-day forecast model, a large dataset means nothing but a lot of samples within a regression problem. So if would be a bad idea trying to infer the uncertainty just from the samples (as it is unfeasible to sample the entire feature space....).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.