3
$\begingroup$

I have ten years of data, which I have averaged. Is it possible to calculate the probability that a future point we be less than a threshold based on my data?

For example, say the average is 0.065. How can I calculate the probability, P, that a future data point will be less than 0.1? I am looking at daily data points, where each day is an average of the daily data for the previous ten years.

I understand I may need something other than the average, like a distribution it something, but I do not know what that is. Depending on how specific an answer is, I can look up equations and formulas.

Thanks.

$\endgroup$
  • $\begingroup$ Can you tell from which distribution are your samples? $\endgroup$ – gt6989b May 25 '17 at 14:38
  • $\begingroup$ Daily percent over ten years. $\endgroup$ – user1187621 May 25 '17 at 15:34
1
$\begingroup$

You can get some useful data if you assume that your 10 years of daily data are independant and of identical distribution.

In that case, your best estimator of the probability of a measurement to be lower than 0.1 is :

$\frac{n}{N}$

where $n$ is the number of observations lower than $0.1$ and $N$ the total number of observations.

Yes, it's that simple. Please be aware that the devil is in the hypothesis, as usual in statistics.

More complex models would try to derive the distribution of your observations in a fixed set of distributions.

Even more complex model would state that the observations are not independant and identically distribued, but this is a much wider area.

$\endgroup$
  • $\begingroup$ Thank you. Just a point of clarification - If we call each daily data point a random %, I believe that the "independent and identical distribution" holds. Correct? $\endgroup$ – user1187621 May 25 '17 at 19:26
  • $\begingroup$ Not necessarily, for example the level of the S&P index in 1 year is random, it's level in 1 year and 1 day is random, but both are not independant. If you ask me "What is the probability that tomorrow S&P index is below 1000", I'd be much smarter looking at today's value than at 10 years historical data. $\endgroup$ – WNG May 25 '17 at 19:31
  • $\begingroup$ Without going into too much detail (for proprietary purposes), this is weather data. I know that there are parameters that contribute to the weather, but the weather on January 1, 2011 could be considered independent of the weather on January 1, 2010 (I understand that one could make a connection between the two, but for all intents and purposes it could be considered independent). $\endgroup$ – user1187621 May 25 '17 at 19:35
  • $\begingroup$ if it's weather data, you will have a lot of autoregressive behaviour, and also seasonality. The approach aforementioned will be very weak. You should check on Cross Validated because chosing the statistical model will actually be the most difficult part of your work. $\endgroup$ – WNG May 25 '17 at 19:38
  • $\begingroup$ I'm intentionally ignoring things like temperature, moisture, surface temperature, solar radiation, time of day, etc. for simplicity. I'm trying to go from "the estimation is 10%" to "the prediction is 95% chance that it will be less than 10%", all based on 10-years of historical data for the single data point I am looking at. $\endgroup$ – user1187621 May 25 '17 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.