Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Currently I am measuring data (Counts over Time).

Due to measurement problems I have some nasty peaks in this data. These peaks are periodical, very sharp (~3 datapoints over a range of 10000) and about 3 times higher than the normal noise ('delta-peaks'):

......|......|......|......|......|......

Currently we just clear these values out by setting these to NaN (not a number).

I think this is a typical solution by physicists and you would find a more elegant way.

I thought about Fourier filter, but since these peaks are very sharp the Fourier transform has periodic 'delta-peaks', too.

Do you have an idea how to solve this problem?

Thanks a lot.

share|improve this question
1  
Is there any value in the data in these peaks? For example, if one was 4 times higher than normal noise, would you believe the data is higher than normal? If not, setting it to NaN is fine. If yes, are the peaks reproducible? If so, you could just subtract them from the data. –  Ross Millikan Oct 8 '12 at 2:52
    
if I understand you wright (I am not sure of that): actually I don't know what these peaks do with my data. In my spectrum I have some other (real) sharp peaks, too. And these real peaks are overlapped with these periodic peaks. I can only calculate the 'delta-peak'-height-to-noise-ratio but not the 'delta-peak'-to-real-peak-ratio. Setting the 'delta-peaks' to NaN should be the same like mean over the noise. But I think that there could be information in the 'delta-peaks' –  willkuer Oct 8 '12 at 4:09
    
If the peaks are noise that is vastly greater than the signal you can't learn anything looking at the values in those bins. If it is just additive and a repeatable amount, you can. In the limit, the noise addition is (say) 5,20,5 counts in three successive bins every 100. Then you can just subtract this from your data. The point is that if you have a good model of the noise, you can subtract it. If not, not. –  Ross Millikan Oct 8 '12 at 4:18
    
actually I think the delta-noise is not that simple. I think that the peak height of these peaks is dependent on the change in the signal. The ratio and difference for the constant background between delta peak and noise is not the same as the ratio and difference between real peaks and delta peaks in the signal. Conclusion: I really don't know how the model would look like. That's why I wanted to apply a filter that is not directly depending on the height of the delta-peaks –  willkuer Oct 9 '12 at 1:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.