# How to interpret the results of a 2 sample KS-test

I have some data which I want to analyze by fitting a function to it. To do that, I have two functions, one being a gaussian, and one the sum of two gaussians. To test the goodness of these fits, I test the with scipy's ks-2samp test. The result of both tests are that the KS-statistic is $0.15$, and the P-value is $0.476635$.

There is clearly visible that the fit with two gaussians is better (as it should be), but this doesn't reflect in the KS-test.

I am sure I dont output the same value twice, as the included code outputs the following: (hist_cm is the cumulative list of the histogram points, plotted in the upper frames)

KStest= stats.ks_2samp(hist_cm,hist_fit_cm)
KStest_1gauss = stats.ks_2samp(hist_cm,hist_fit_1gauss_cm)
print KStest
print KStest_1gauss
#output
#(0.15000000000000002, 0.47663525071642981)
#(0.15000000000000002, 0.47663525071642981)


I know the tested list are not the same, as you can clearly see they are not the same in the lower frames.

So i've got two question: Why is the P-value and KS-statistic the same? (this might be a programming question). And how to interpret these values?

edit: I calculate radial velocities from a model of N-bodies, and should be normally distributed. In the figure I showed I've got 1043 entries, roughly between $-300$ and $300$. I then make a (normalized) histogram of these values, with a bin-width of 10. To this histogram I make my two fits (and eventually plot them, but that would be too much code)

bins = np.arange(-300,301,10)
hist, bin_edges = np.histogram(v[i], normed=True, bins=bins)
hist_cm=np.cumsum(hist)
bin_centres = (bin_edges[:-1] + bin_edges[1:])/2
coeff, pcov2 = leastsq(residuals, x0=(0.01,0.,60.,0.01,150.,40.) ,args=(bin_centres, hist))
coeff, pcov2 = leastsq(residuals, x0=(0.01,0.,60.,0.01,150.,40.) ,args=(bin_centres, hist))
hist_fit = gauss2(bin_centres, *coeff)
hist_fit_cm=np.cumsum(hist_fit)
coeff_1gaus, pcov2_1gauss = leastsq(residualsOneGauss, x0=(0.01,0,100), args=(bin_centres, hist))
hist_fit_1gauss = gauss(bin_centres, *coeff_1gaus)
hist_fit_1gauss_cm = np.cumsum(hist_fit_1gauss)

#and the used functions:
def gauss(x, A, mu, sigma):
return A*np.exp(-(x-mu)**2/(2.*sigma**2))

def gauss2(x,A, mu, sigma, A2, mu2, sigma2):
if A2<0:
return 1000
return A*np.exp(-(x-mu)**2/(2.*sigma**2))+ A2*np.exp(-(x-mu2)**2/(2.*sigma2**2))

def residuals(p, x,y):
integral = quad( gauss2, -500, 500, args= (p[0],p[1],p[2],p[3],p[4],p[5]))[0]
penalization = abs(1-integral)*1000
return y - gauss2(x, p[0],p[1],p[2],p[3],p[4],p[5] ) - penalization

def residualsOneGauss(p,x,y):
integral = quad( gauss, -500, 500, args= (p[0],p[1],p[2]))[0]
penalization = abs(1-integral)*1000
return y - gauss(x, p[0],p[1],p[2]) - penalization

• Are your distributions fixed, or do you estimate their parameters from the sample data? In the latter case, there shouldn't be a difference at all, since the sum of two normally distributed random variables is again normally distributed. Also, why are you using the two-sample KS test? That's meant to test whether two populations have the same distribution (independent from what the specific distribution is), not to check whether one polulations has a specific distributions. For the latter case, the one-sample KS test is appropriate, but only if the distribution to compare to is fixed. – fgp Apr 9 '14 at 11:38
• I estimate the variables (for the three different gaussians) using scipy.leastsq with a penalty on the integrand. I dont get why I get the same results with both functions. What is another way to test my fit with the provided data? – Mathias711 Apr 9 '14 at 11:43
• That doesn't make sense. If you estimate the parameters, i.e. determine the gaussian's mean and variance, from the sample data, than what's the difference between your three different gaussians? – fgp Apr 9 '14 at 11:46
• Because I have two functions: func1 is the sum of two, uncorrelated (the parameters arent, the function of course is), gaussians. And func2, that is just  gaussian. In my picture, in the lower left frame, you see in green and red the two indepent gaussians from func1, and in teal the sum of these two, which is func1`. I hope this made it clearer – Mathias711 Apr 9 '14 at 11:50
• I've said it, and say it again: The sum of two independent gaussian random variables is again a gaussian random variable! If $X_1,X_2$ have means $\mu_1,\mu_2$ and variables $\sigma_1^2,\sigma_2^2$, then $X_1 + X_2$ has mean $\mu_1 + \mu_2$ and variance $\sigma_1^2 + \sigma_2^2$. The fact that you're see a difference at all (even if not in the KS test statistic) is just an artifact of the way you estimate the parameters, I think. – fgp Apr 9 '14 at 11:53