To begin with, there is no such thing as the 'set of all groups.' What you are looking for is a proper class, for which this question should be helpful.
In response to the part about $\chi$ being injective, this is surely not true, as many groups have the same automorphism group - the trivial group and $\mathbb{Z}_2$, for a simple example.
An interesting paper by Iyer contains numerous other examples of groups with the same automorphism groups, in particular Prop. 6.1 - 6.6. He also shows that a nonabelian simple group has the same automorphism group as its covering group.
Furthermore $\chi$ is not surjective. For example, $\mathbb{Z}_p$ is not an automorphism group of any group for $p>2$. (Jack Schmidt says it best here.) Additionally, there is no group $G$ such that $\text{Aut}(G)$ is an Abelian $p$-group of order $\leq p^{11}$ for $p>2$. (reference.) It is also worth noting that Iyer proves that $S_6$ is not the automorphism group of any finite group, and that every finite group occurs as the automorphism group of at most finitely many finite groups.