# Determine which mean is smaller over two non-normal distributions

Let's say I have a non-normal distribution A and another non-normal distribution B, the mean and std deviations of each distribution are different.

I then randomly sample 100 values from A, SampleA, and 50 values from B, SampleB.

Given only SampleA and SampleB, what is the equation to determine the probability that the mean of A is less than the mean of B.

I'm not a statistician, if this problem is underspecified leave a comment and I'll update it with any relevant information.

I've looked into t-tests, but things I've read have made it sound like it is inapplicable to non-normal distributions. I'm also unsure why I care about things like the null hypothesis and 95% confidence intervals when I'm only concerned with the specific probability that one distribution has a lower mean than the other.

-
You want the Mann-Whitney U test: en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U – Michael Lugo Jul 16 '13 at 23:02
I would do a quick sampling to estimate the probability. Do you need theoretical values? – Memming Jul 16 '13 at 23:09

Why not resample?

Without getting into the difficulties of what you describe as "the probability that the mean of A is less than the mean of B," suppose you take 10,000 resamples (with replacement) of size 100 from sampleA and size 50 from sampleB. Now construct the distribution of the differences of their sample means.

As an example, suppose 97% of the differences have the sample mean from sampleA being greater than the sample mean from sampleB. What could you conclude? How confident are you of this conclusion?

-

(1) Re-sampling is certainly a possibility, as suggested in a previous answer. Also. possibly a permutation test or a bootstrap confidence interval, which are different kinds of resampling than suggested there. (If you can figure out how to do one of these, this might be your best bet.)

(2) If there are few (if any) ties within or between the two samples, a Mann-Whitney test (equivalent to a Wilcoxon rank sum test) would be a possibility. However, if there are many ties, such rank-based tests lose power and require adjustments.

(3) Because you have relatively large sample sizes, unless the populations are far from normal, you can use a t-test (the Welch 'separate variances' version would be best) for approximate results. The degrees of freedom will probably be large, so the distribution of the T statistic will be close to normal. By 'approximate results', I mean that the P-value will be only approximate, so I would feel uncomfortable rejecting the null hypothesis unless the P-value is less than 1% or so.

Notes: (a) The Welch T-statistic essentially provides the answer to your original question. For your sample sizes it should be nearly normal and you will reject that the two population means are the same if the absolute value of the T-statistic exceeds 2 (nominal 5% level).

(b) Most statistical software does the 'Welch' two-sample t test by default, doing the 'pooled' test only if you insist.)

Addendum: As an illustration of the behavior of markedly nonnormal distributions, I simulated 100 observations from the strongly skewed distribution Gamma(4, rate=1/4), which has population mean 16, SD 8. My sample had $\bar X = 15.78$ and $S_X = 8.07$. I also generated 50 observations from the distribution Unif(0, 50), which has population mean 25 and SD 14.43 (and no tails). My sample had $\bar Y = 22.34$ and $S_Y = 13.9.$