I have some EXTREMELY noisy data (standard deviation a is greater than the mean), but plotting it with a 15 data point running average does well to get a visual indication of the trending. I want to see if any of my data sets are correlated, but standard correlation gives me very poor results due to the noisiness of the data (and the fact that I care about trends over scales on the order of hundreds of data points). I'm thinking of maybe just cranking the running average up to a few hundred data points and then running the correlation on those, but something makes me think that there might be a better way.
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Your use of a "running average" sounds like you have some kind of time series data. If so may want to look at cointegration rather than correlation to analyze it. You can find a good discussion of the differences between correlation and cointegration on SE's Quantitative Finance Beta site. See:How are correlation and cointegration related?
Not original to me, but the simple comparative description in the mentioned post that best captures the difference for me came from the contributor, isomorphismes:
and a comment:
from user, Gravitas.
If you still want to stay with correlation consider applying it to the log of the differences at each time step.