Kolmogorov–Smirnov 2-sample test with large sample sizes always significant I'm aware that the probability of a traditional statistical test such as student's t or mann-whitney u being deemed significant approaches 1.0 as sample size increases (i.e. >10,000) but I'm getting the same issue with a Kolmogorov–Smirnov 2-sample test which I'm having trouble understanding. Doesn't it always evaluate the difference between two sets of 100 cumulative probability values? I don't understand how sample size affects the result.
 A: Comment continued:
I generated two samples ($X$ and $Y$) of $n = 5000$ each from $\mathsf{Norm}(100, 15)$ and
a third ($Z$) of the same size from $\mathsf{Norm}(100, 14).$ 
Results from the K-S test in R statistical software (where 5000 is the size limit)
are as shown below. (With a different seed you would get somewhat different
samples, hence somewhat different results.)
set.seed(1017)
x = rnorm(5000,100,15);  y = rnorm(5000,100,15);  z = rnorm(5000,100,14)
ks.test(x, y);  ks.test(x, z);  ks.test(y, z)


        Two-sample Kolmogorov-Smirnov test
data:  x and y 
D = 0.0126, p-value = 0.8222           # Same population, not rejected
alternative hypothesis: two-sided 


        Two-sample Kolmogorov-Smirnov test

data:  x and z 
D = 0.0242, p-value = 0.107            # slightly different pop's, nearly rej
alternative hypothesis: two-sided 


        Two-sample Kolmogorov-Smirnov test

data:  y and z 
D = 0.0312, p-value = 0.01539          # slightly different pop's, rejected
alternative hypothesis: two-sided 

Boxplots of the examples are very similar, except perhaps for fewer, less-extreme outliers in the third.:

Below the ECDFs of each sample are compared with the CDF of $\mathsf{Norm}(100, 15)$ (heavy, dashed curves). In the third plot maybe you can see a slight discrepancy.

