Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Given a function $f\left(x\right)$, I have two formulas to compute the coefficients of the same harmonic series approximation to $f\left(x\right)$. Call the results of each formula $^1c_k$ and $^2c_k$ for formula 1 and 2 respectively.

I want to test whether the two formulas give the same result for all $k$. Assuming there is no analytical way to test for equivalence, I wanted to perform some kind of statistical hypothesis test.

So far I have computed $^1c_k$ and $^2c_k$ for hundreds of different functions: $f_1\left(x\right), f_2\left(x\right),\ldots,f_n\left(x\right)$. So that for each $k$ I have a set of paired values for $^1c_k$ and $^2c_k$:

$$^1C_k=\left\{^1c_k\left(f_1\left(x\right)\right),^1c_k\left(f_2\left(x\right)\right),\ldots,^1c_k\left(f_n\left(x\right)\right)\right\}$$

and

$$^2C_k=\left\{^2c_k\left(f_1\left(x\right)\right),^2c_k\left(f_2\left(x\right)\right),\ldots,^2c_k\left(f_n\left(x\right)\right)\right\}$$

My original hypothesis to test was that the error between the results of either formula should be normally distributed about zero. My first inclination was to use a paired t-test, but the paired errors fail normality tests (to varying degrees depending on which normality test you use). I've tried rank-sum tests that didn't give great results. So it looks like the errors are not always normally distributed about zero. But they are generally close to zero.

One reason that they might not be normally distributed is that one of the formulas might give a result that approaches the result of the other formula asymptotically (i.e. one is always or usually below the other), this would fail my original hypothesis, and yet suggest that the results are the same in the limit of increasing accuracy.

I am now considering using convergence acceleration (e.g. Wynn's epsilon algorithm) to compare the asymptotic values of each formula for each $k$, but this doesn't really give any measurement of the statistical significance of any difference that I might find, nor provide any error bars on the conclusions I arrive at. So I'm still at a loss for how to test statistically whether the two formulas produce equivalent results, any ideas?

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.