Are these permutation groups, defined by asymptotic properties, isomorphic? For a permutation $\sigma : \mathbb{N} \to \mathbb{N}$, let $c_{\sigma}(n) = |\{k \in [n] \mid \sigma(k) \ne k\}|$, and for a function $f : \mathbb{N} \to \mathbb{R}$, let $G_f = \{\sigma \in S_{\mathbb{N}} \mid \lim_{n \to \infty} c_\sigma(n)/f(n) = 0 \}.$ For example, if $f(n) = n$, then $G_f$ is just the permutations of $\mathbb{N}$ that fix almost everything, and if $f(n) = n^2$ (or something else that grows much faster than $n$) then $G_f$ is just all permutations of $\mathbb{N}$. My question is are the groups $\{ G_{x^r} \mid r \in (0,1]\}$ all isomorphic? Isomorphic to all permutations of $\mathbb{N}$? I'm able to see that they mutually contain each other, but that isn't enough to resolve this. Have people studied groups that are defined in such ways?
This question is inspired by Is there a natural family of nonisomorphic groups parametrized by $\mathbb{R}$?, as showing my groups aren't isomorphic would resolve that question nicely. If my groups are isomorphic, I'm also curious if there are computably isomorphic in the sense that given a list of what my input permutation does I can access sequentially, I can begin to compute what the permutation it maps to does.
edit: Ycor's answer below shows that if they are isomorphic, they are conjugate, but I'm still very curious about the outcome: as mentioned in that answer, being conjugate would require a permutation with pretty surprising properties, and ruling out the existence of such a thing (or showing there is one!) seems a worthy problem in its own right. Note (as mentioned in a comment) Ycor's partial result rules out any of these groups being isomorphic to all permutations of $\mathbb{N}$
 A: (I always find depressing to see group theory classified as "abstract" algebra at some places.)
It's correct they're pairwise non-isomorphic (namely, the groups $H_r=\{\sigma\in S_\mathbf{N}: c_\sigma(n)=o(n^r)\}$ for $r\in \mathopen]0,1]$, denoted $G_{x^r}$ in the question).
The first observation is that they're not conjugate: this is easy (using that for every permutation $f$ of $\mathbf{N}$ there's an infinite subset $I$ such that $f(n)\ge n$ for all $n\in I$.

Edit this non-conjugation claim is unclear at this point. So for the moment this post reduces showing that they're non-conjugate, and it turn this means: for $0<r<s\le 1$, does there exist a permutation $f$ of $\mathbf{N}$ such that $f(N_r)=N_s$, where $N_u$ is the set of subsets $I$ of $\mathbf{N}$ that are $n^u$-negligible: $\#(N_u\cap [n])=o(n^u)$

Next, we see that an isomorphism between two such groups would be implemented by a conjugation. Namely we have the following lemma (which is probably a very particular result of M. Rubin's theorems):
Lemma let $X$ be an infinite set; let $U,V$ be subgroups of $S_X$ that contain the group $S_X^\#$ of finitely supported permutations of $X$. Let $f$ be an isomorphism $U\to V$. Then $f$ is the restriction of a conjugation of $S_X$: there exists $\alpha\in S_X$ such that $f(g)=\alpha g\alpha^{-1}$ for all $g\in U$. In particular, $V=\alpha U\alpha^{-1}$.
To prove the lemma, the first step is to check that $f(S_X^\#)=S_X^\#$: namely we need to "recognize" $S_X^\#$ within both $U$ and $V$. This is done in "Lemma*" below.
The second step is a classical one: $\mathrm{Aut}(S_X^\#)=S_X$ (acting by conjugation). This can be found in the books by Scott or Dixon-Mortimer. So, there exists $\alpha\in S_X$ such that $f(w)=\alpha w\alpha^{-1}$ for all $w\in S_X^\#$.
The third and concluding step: for $g\in U$ and $w\in S_X^\#$, we compute $f(gwg^{-1})$ in two ways: first it equals $f(g)f(w)f(g)^{-1}=f(g)\alpha w\alpha^{-1}f(g)^{-1}$, second it equals $\alpha gwg^{-1}\alpha^{-1}$. Since the centralizer of $S_X^\#$ is trivial, we deduce $\alpha g=f(g)\alpha$ for all $g\in U$, which is the desired form.
Lemma$^*$ Let $U$ be a subgroup of $S_X$ containing $S_X^\#$. Let $T_U$ be the set of elements $s$ of order 2 in $U$ such that for every $U$-conjugate $t$ of $s$, either $t$ commutes with $s$ or $(st)^3=1$. Then $T_U$ is the set of transpositions of $X$. In particular, $\langle T_U\rangle=S_X^\#$ is a characteristic subgroup of $U$, and for any two such groups $U_1,U_2$, every isomorphism $U_1\to U_2$ induces an automorphism of $S_X^\sharp$.
It is clear that a transposition satisfies this property. Conversely, suppose that $s$ has order 2 and is not a transposition. Say, $s(a)=b$ and $s(c)=d$ with $|\{a,b,c,d\}|=4$.
First case, $s$ fixes at least two points. Then there exists a $S_X^\#$-conjugate $t$ of $s$ fixing $a,d$ and exchanging $b,c$. Then $st$ has order 4 on $\{a,b,c,d\}$.
Second case, $s$ fixes at most one point. Then there exist $e,f,g,h$ such that $s(e)=f$ and $s(g)=h$, and $|\{a,b,c,d,e,f,g,h\}|=8$. Hence there exists a $S_X^\#$-conjugate of $s$ exchanging all pairs $\{b,c\}$, $\{d,e\}$, $\{f,g\}$, $\{h,a\}$. Then $st$ has order 4 on $\{a,b,c,d,e,f,g,h\}$.
In both cases, we see that $t$ and $s$ don't commute and $(st)^3\neq 1$.
A: A proof that the groups $G_{x^r}$ are not conjugate, which proves that they are not isomorphic by YCor's answer:
Lemma: For any permutation $\alpha : \mathbb{N} \to \mathbb{N}$, and any $t \in (0, 1]$, there is a set $B \subset \mathbb{N}$ with $|B \cap [n]| = O(n^t)$ and $|\alpha(B) \cap [n]| = \Omega(n^t)$.
Proof: For $k \geq 0$, note that the set $A_k = \alpha^{-1}([2^{k+1}]) \setminus [2^k]$ has size at least $2^k$, so let $B_k$ be a subset of $A_k$ of size $2^{kt}$, and define $B = \bigcup_k B_k$. Then for $2^k < n \leq 2^{k+1}$, all $B_j$ with $j \geq k+1$ are disjoint from $[2^{k+1}]$, hence
$$B \cap [n] \subset B \cap [2^{k+1}] \subset \bigcup_{j=0}^k B_j$$
which gives the bound $|B \cap [n]| \leq \sum_{j=0}^k 2^{jt} \leq C \cdot 2^{kt} \leq Cn^t$, meaning $|B \cap [n]| = O(n^t)$. In the other direction, for $n \geq 2$ we have $2^k \leq n < 2^{k+1}$ for some $k$, hence since $\alpha(B_{k-1}) \subset \alpha(B) \cap [2^k]$ this gives
$$|\alpha(B) \cap [n]| \geq |\alpha(B) \cap [2^k]| \geq |\alpha(B_{k-1})| = 2^{(k-1)t} \geq 2^{(k+1)t - 2} \geq \frac{1}{4} n^t$$
so $|\alpha(B) \cap [n]| = \Omega(n^t)$.
Now, suppose there is an isomorphism $f$ from $G_{x^s}$ to $G_{x^r}$, for some $0 < r < s \leq 1$. By YCor's answer, $f$ takes the form $f(g) = \alpha g \alpha^{-1}$ for some permutation $\alpha : \mathbb{N} \to \mathbb{N}$. Take $t$ with $r < t < s$, and let $B \subset \mathbb{N}$ be as in the Lemma. Now let $\pi$ be a permutation $\mathbb{N} \to \mathbb{N}$ with $\{k \in \mathbb{N} \mid \pi(k) \neq k\} = B$; it is straightforward to construct such a permutation. We have $c_\pi(n) = |B \cap [n]| = O(n^t) = o(n^s)$, so $\pi \in G_{x^s}$, and thus $\sigma := f(\pi)$ is in $G_{x^r}$. But $\sigma = \alpha \pi \alpha^{-1}$, so $\{k \in \mathbb{N} \mid k \neq \sigma(k)\} = \{\alpha(k) \mid k \neq \pi(k)\} = \alpha(B)$, so $c_\sigma(n) = |\alpha(B) \cap [n]| = \Omega(n^t)$, which contradicts $\sigma \in G_{x^r}$. Thus $G_{x^r}$ and $G_{x^s}$ are not isomorphic.
