Recovering a group action from sizes of orbits of individual elements

Question

Let $G$ be a group (say, finite) and let it act on a set $X$ (say, also finite). For every element $g \in G$, we can consider its action on $X$. My rather vague question is

What information about the sizes of orbits of $X$ under $G$, can we recover only from knowing $G$ and the sizes of orbits of $X$ under $g$ for each $g\in G$?

Perhaps a better phrasing is what information we can't recover if any. An example of two actions, that shows we can't recover everything will be a good start.

One simple observation is that Burnside's lemma shows that $|X/G|$ is the average of the number of fixed points of $g \in G$. Hence, this piece of information can be recovered (even without knowing the group structure, which is available to us). The question is, what else. I am mainly interested in $X^G$, the number of fixed points of $X$ under the whole of $G$.

One last remark. What I described amounts to saying that we know the action of every cyclic subgroup of $G$ and we want to recover (as much as possible from) the action of $G$. Perhaps we should look at a slightly bigger family of subgroups for this.

Answer 1

Let $G=\{1,a,b,ab\}$ be a Klein $4$-group, and consider the following two actions on $\{1,2,3,4,5,6\}$.

In the first action

$a \mapsto (1,2)(3,4)$, $b \mapsto (1,3)(2,4)$, $ab \mapsto (1,4)(2,3)$,

and in the second action

$a \mapsto (1,2)(3,4)$, $b \mapsto (3,4)(5,6)$, $ab \mapsto (1,2)(5,6)$.

In both actions, all three involutions have two orbits of length $2$ and two of length $1$. In the first action, the orbits of $G$ have lengths $4,1,1$, and in the second action they are $2,2,2$.