# Structure of the automorphism group of a group

I'm looking to represent the automorphism group of a finite group in a small group theory package I'm writing.

I was thinking of representing a generic automorphism $$a \in \operatorname{Aut}(G)$$ as a pair $$(o,i)$$ representing the composition of an outer automorphism $$o \in \operatorname{Out}(G)$$ by an inner automorphism $$i \in \operatorname{Inn}(G)$$. Thus I can write $$i$$ as an element of $$G / \operatorname{Center}(G)$$, and use a set of coset representatives for $$\operatorname{Out}(G)$$. With this information, I can at least enumerate the elements of $$\operatorname{Aut}(G)$$.

Now, I'd like eventually to use the group structure on $$\operatorname{Aut}(G)$$. I guess that I'd need to write $$\operatorname{Aut}(G)$$ as a semidirect product to be able to compute the composition of elements of $$\operatorname{Aut}(G)$$ as represented by those pairs. Now, as some automorphism groups do not split in that way, I guess I'm out of luck?

I'm not so familiar with the group extension problem.

Yes, you are indeed out of luck when that extension is nonsplit! Representing $${\rm Aut}(G)$$ for finite groups $$G$$ is one of the more difficult problems in computational group theory.
The default approach is to look for as small a subset of $$G$$ as you can find (unions of conjugacy classes are a good thing to try) on which $${\rm Aut}(G)$$ acts faithfully, and to use that to get a permutation representation of $${\rm Aut}(G)$$, but in difficult examples that can lead to very large degrees.
For finite simple groups $$G$$, it is useful to store suitable representation of $${\rm Aut}(G)$$ - so for example, for $$A_n$$ with $$n \ne 6$$ you can use $$S_n$$.