# How to decide the randomness of a dataset by checking the prime numbers inside it?

So it is weekend! I am reading currently a book where I found this sentence: "71 percent of men said they had a 'good sense of direction'. Only 47 percent of women reported same thing.", and I thought 'nonsense! two primes in a row, this stastistic is a fake!'. Ok it might be exaggerated, but then I started to think about the question I am asking now... How could I decide when a supposed random dataset is not so random by looking at the prime numbers inside?

I would do as follows, but I am not sure if it would be right or not, so I would like to read other solutions.

To simplify, let us imagine a dataset of 100 random different integer elements of any range.

$S=\{567,10,...,10001,205,8747\}$

First, I would reorder the dataset from the smaller to the greater.

$S=\{10,205,567...8747,10001\}$

a) Then if there is randomness, I would expect the random primes to be gradually more scattered to the right of the dataset, and less scattered at the beginning of the dataset in some extent (as it happens with the distribution of primes into the integers). If that does not happens, then I would think the dataset is not so random.

b) I would compare $(1)$ $\frac{S_{last}-S_{first}}{\pi(S_{last})-\pi(S_{first})}$ versus $(2)$ $\frac{Total-elements-of-S}{Total-primes-in-S}$.

If $(2)\gt(1)$ I would think that it is not so random.

Would this be right or not? I would like to know other options, thank you!

## 2 Answers

(2) > (1) is far too strict. Even a perfectly random set would behave this way half the time.

You need to have some notion of variance. A simplistic strategy to start with would be to use the Cramér model where each integer $n \ge 2$ is attached to a random indicator variable $X_n$ which is $1$ with probability $1/\ln n$. Then you can extract the expected value and variance of $\sum_{n \in S} X_n$ and compare the actual number of primes to this. More sophisticated would be to work out the actual tail probabilities for the random variable $\sum_{n \in S} X_n$ to determine a $p$ value.

Another line of attack would be to study the dataset modulo "small" primes. A random set would be fairly equidistributed in each case (not sure how small is "small" enough). If the set sieves well over many small primes then it seems likely to possess the expected number of primes naturally, rather than looking specifically for primes.

• thank you very much, I will review this in deep, I am not so familiar with the Cramer model. The idea of using the congruences of small primes is very cool. – iadvd Mar 28 '15 at 3:32

You may be interested in looking into Benford's Law - it's allows you to consider the randomness of many types of data based on the incidence of each digit in the leading position. Pretty interesting stuff- related to logs and exponential growth

• Note that a "random subset" of the primes tends to obey Benford's Law as well, but it has way more than the usual number of primes :). – Erick Wong Mar 28 '15 at 3:27
• @Mike Andrejkovics thank you for the reference, to be honest it is the first time I hear about Benford's Law. – iadvd Mar 28 '15 at 3:34