Groups acting on groups When a group acts on a set, many doors open up. The orbits partition the set, the stabilizers are subgroups, the size of an orbit is the index of the stabilizer, the # of orbits is the average # of fixed points, etc. But when a finite group acts on a finite-dimensional complex vector space, the additional structure reveals even more information about the action. For example, the representation decomposes into irreducible subrepresentations that can be described in a character table.
I would suspect that if a group $G$ acts on a group $H$ in a sensible way, then one can derive additional information beyond the possibilities of a group acting on a set, just like in representation theory. By "sensible way" I mean that the action satisfies the additional property $g * (h h') = ( g * h) (g * h')$ for all $g \in G$ and $h, h' \in H$, which gives a group homomorphism $G \to \mathrm{Aut}(H)$. One example is $G$ acting on itself by conjugation. If $H$ is an abelian group, then 2021 Wikipedia would call this sort of action (together with $H$) a $G$-module, although I think there are different uses of that term.
One consequence of the new property is that
$$g * 1_H = g * (1_H 1_H) = (g * 1_H)(g * 1_H) \in H \text, $$
so $g * 1_H = 1_H$ for all $g \in G$; $1_H$ is always a fixed point of the action.

What are some properties of groups acting on groups that are not always true of groups acting on sets?


Imposing too many constraints can make the phenomenon bland. If $g * (h h') = (g * h)h'$ for all $g \in G$ and $h, h' \in H$, then $g * h = g * (1_H h) = (g * 1_H)h = 1_H h = h$ for all $h \in H$, so the action is trivial. This is related to Notions of groups acting on groups, although that question relinquishes the assumption that $g * (h h') = (g * h) (g * h')$.
 A: If a group $G$ acts on another group $H$ (by automorphisms), then a distinguishing condition takes place, namely:
$$\operatorname{Fix}(g)\le H, \forall g\in G \tag 1$$
where $\operatorname{Fix}(g):=\{h\in H\mid \phi_g(h)=h\}$, being $\phi\colon G\longrightarrow \operatorname{Aut}(H)$ the action. In fact, for every $g\in G$, $\phi_g(1_H)=1_H$, whence $1_H\in\operatorname{Fix}(g)\ne\emptyset$; moreover, by definition, $\operatorname{Fix}(g)\subseteq H$; finally, for every $g\in G$ and $h_1,h_2\in\operatorname{Fix}(g)$, $\phi_g(h_1h_2^{-1})=$ $\phi_g(h_1)\phi_g(h_2^{-1})=$ $\phi_g(h_1)\phi_g(h_2)^{-1}=$ $h_1h_2^{-1}$, whence $h_1h_2^{-1}\in\operatorname{Fix}(g)$.
If $G$ and $H$ are both finite, say $G=\{g_1,\dots,g_{|G|}\}$ and $H=\{h_1,\dots,h_{|H|}\}$, then the usual equation:
$$\sum_{i=1}^{|H|}|\operatorname{Stab}(h_i)|=\sum_{j=1}^{|G|}|\operatorname{Fix}(g_j)| \tag 2$$
and the usual condition (by Burnside's Lemma):
$$|G|\mid \sum_{j=1}^{|G|}|\operatorname{Fix}(g_j)| \tag 3$$
give new opportunities, according to the fact that -by Lagrange's Theorem- the summands in the RHS of $(2)$ and in $(3)$ must all divide $|H|$.
As an entry test for this new setting, let's take $G=C_p$ and $H=C_q$, where $p$ and $q$ are distinct primes. If a nontrivial homomorphism exists, then $|\operatorname{Fix}(g_{\bar j})|=1$ and $|\operatorname{Stab}(h_{\bar i})|=1$, for some $\bar j\in \{1,\dots,p\}$, $\bar i\in\{1,\dots,q\}$. Then $(2)$ and $(3)$ yield:
$$[\space k+(q-k)p=l+(p-l)q\Longrightarrow k(p-1)=l(q-1)\space]\space\wedge\space [\space p\mid pq-l(q-1)\space ] \tag 4$$
for some $1\le k\le q$ and $1\le l\le p$. Now:

*

*if $p>q$, then from $(4)$-2nd term of the "$\wedge$": $p\mid pq-l(q-1)\Longrightarrow$ $p\mid l \Longrightarrow$ $l=p\Longrightarrow$ $k(p-1)=p(q-1)\Longrightarrow$ $k=\frac{p}{p-1}(q-1)>q-1\Longrightarrow$ $k=q$; but $(k,l)=(q,p)$ is not a solution of $(4)$-1st term of the "$\wedge$": so, for $p>q$, there are no nontrivial homomorphisms $\phi\colon C_p\longrightarrow\operatorname{Aut}(C_q)$;

*if $p<q$ and $p\nmid q-1$, then from $(4)$-2nd term of the "$\wedge$": $p\mid pq-l(q-1)\Longrightarrow$ $p\mid l$, and we fall back into the previous case.

Therefore, if $p\nmid q-1$, there are no nontrivial homomorphisms $\phi\colon C_p\longrightarrow\operatorname{Aut}(C_q)$. This hasn't used any knowledge about the isomorphism class of the group $\operatorname{Aut}(C_q)$. (Incidentally, that for $p\mid q-1$ there is actually a nontrivial homomorphism $\phi$, it is shown e.g. here.)
As one more application, let's now assume $G=C_p$ and $|H|=q^2$, still with $p,q$ distinct primes. If a nontrivial homomorphism exists, then $(2)$ and $(3)$ yield:
$$[\space k+(q^2-k)p=l_0+l_1q+(p-l_0-l_1)q^2\space]\space\wedge\space [\space p\mid k\space ] \tag 5$$
for some $1\le k\le q^2$ and $1\le l_0+l_1\le p$. Since $p\mid k\Longrightarrow p\le k\le q^2$, for $p>q^2$ there are no nontrivial homomorphisms $\phi$. This suffices to give a different proof to, e.g., this question (and the conclusion doesn't change with $\Bbb Z_2\times \Bbb Z_2$ in place of $\Bbb Z_4$). Again, this hasn't used any knowledge about the isomorphism class of the group $\operatorname{Aut}(H)$.
A: Suppose $G, H$ are groups and $\phi : G \to \mathrm{Aut}(H)$ is a group homomorphism. As noted in the post, $\phi$ is equivalent to a sensible action given by $g * h := \phi(g)(h)$ for $g \in G, h \in H$.
Define $H \rtimes_\phi G$ to be $H \times G$ under the operation $(h, g)(h', g') = (h (g * h'), gg')$, so that $H \rtimes G$ is a group (the external semidirect product). Write $H_0, G_0$ for the copies of $H, G$ in the group $H \rtimes G$. Then $H_0, G_0$ are subgroups of $H \rtimes G$ with trivial intersection, $H_0$ is normal, and $H_0 G_0 = \langle H_0, G_0 \rangle = H \rtimes G$.
Conversely, if $H_1, G_1$ are subgroups of a group such that $H_1 \cap G_1 = \{ 1\}$, $H_1$ is normal, and $\langle H_1, G_1 \rangle$ equals the group, then the group is an internal semidirect product $H_1 \rtimes G_1$, isomorphic to $H_1 \rtimes_\psi G_1$ for $\psi : G_1 \to \mathrm{Aut}(H_1)$ corresponding to the conjugation action of $G_1$ on $H_1$: $\psi(g) = [h \mapsto ghg^{-1}]$.

In conclusion, a group $G$ acting sensibly on a group $H$ gives rise to a semidirect product $H \rtimes G$, and any semidirect product $H \rtimes G$ implicitly uses such an action to define its multiplication. Therefore the actions of $G$ on $H$ satisfying $g * (h h') = (g * h)(g * h')$ correspond to the possible semidirect products $H \rtimes G$.
