Probability of Relatively Prime Integers

A number theory paper I wrote was recently rejected from a journal due to "working under the untenable hypothesis that the natural density behaves like a probability measure (as it is not $\sigma$-additive and not stable under finite intersections)".

My paper was about providing a lattice-theoretic proof that the probability two Gaussian integers is relatively prime is $\frac{1}{\zeta_{\mathbb{Q}(i)}(2)}$, where $\zeta_{\mathbb{Q}(i)}(2)$ is the Dedekind zeta function over the Gaussian integers (as the known proof by Collins and Johnson from 1989 used difficult analytic number theory techniques). The main objection from the referee was to an argument of the following form:

Theorem: Let $x,y \in \mathbb{Z}$ be randomly chosen. Then, $$P(\gcd(x,y)=1) = \frac{1}{\zeta(2)}$$ Proof: We have that $P(\gcd(x,y)=1) = P(p \nmid x \cap p \nmid y, \forall p)$, where $p$ is prime. For a fixed prime $p$, the probability that $p \mid x$ and $p \mid y$ is $\frac{1}{p^2}$ and so we have that the probability this does not occur is $1 - \frac{1}{p^2}$. Therefore, taking the product over all primes we have that $$P(\gcd(x,y)=1) = \prod_{p \; \text{prime}} \left(1 - \frac{1}{p^2}\right) = \left(\prod_{p \; \text{prime}} \left(\frac{1}{1-p^{-2}}\right) \right)^{-1} = \frac{1}{\zeta(2)}$$

Why is this invalid reasoning?

• The $\gcd$ stuff at the end can be rewritten as an asymptotic density argument. Perhaps so can your Gaussian integer calculation. – André Nicolas Feb 21 '14 at 20:53
• You are using properties of probability. A probability measure on the natural numbers is an assignment of specific non-negative reals $p_k$ such that $\sum_1^\infty p_k=1$. Obviously such a measure cannot assign equal weights to all natural numbers. So for what specific measure are your probabilities calculated? – André Nicolas Feb 21 '14 at 21:53
• That's asymptotic density, don't call the limit, if it exists, probability. Informal arguments that treat it as if it were probability can lead to incorrect conclusions. – André Nicolas Feb 21 '14 at 23:47
• That paper also misuses the terminology! I would not mind it much in the title, but if it is used in the body I would get upset. Yes, if density is used, and existence of asymptotic density (limit) is proved, everything is fine. A bit of history. It had been known for a while that if the limit of $\pi(x)/(x/\ln x)$ exists, that limit must be $1$. The hard part of the Prime Number Theorem was the proof of existence. – André Nicolas Feb 22 '14 at 0:40
• You are welcome. Good luck! Perhaps you ran into a referee in a bad mood. In a better mood, (s)he might have made suggestions about what changes to make. – André Nicolas Feb 22 '14 at 0:45

Say that a positive integer $$n$$ is $$k$$-special if $$n \equiv k \bmod p_k^2$$, where $$p_k$$ is the $$k$$th prime. The density of $$k$$-special positive integers is of course $$1/p_k^2$$.
Now say that $$n$$ is ordinary if $$n$$ is not $$k$$-special for any $$n$$. Applying your reasoning, we get that the density of ordinary positive integers is $$\prod_k \left(1 - \frac{1}{p_k^2} \right) = \frac{1}{\zeta(2)}.$$ But note that $$n$$ is always $$n$$-special. Therefore, there are no ordinary positive integers!
• Why is the density $1/p_k^2$ and not $1/p_k$? – joriki Mar 2 '20 at 17:37
• But it's not just a typo -- your argument relies on it. If you change it to $\frac1{p_k}$, the product diverges, and the density is correctly predicted to be zero. – joriki Mar 3 '20 at 19:05
• You'll see I edited the definition of $k$-special to be mod $p_k^2$, making the argument work. – Mirlan Mar 5 '20 at 1:10