Is an automorphism a function or a group? this is my first time on the Stack Exchange so I apologize if I went about asking this improperly! I was working on a problem set that asked to compute Aut(S3). In class we've been thinking about automorphisms as a function that maps elements of one group to another. However it seems like an automorphism can be better thought of as a subgroup of S3. Could someone provide clarity about what it means to compute/find Aut(S3)?
 A: The automorphism group $\operatorname{Aut} S_3$ is the group of automorphisms on $S_3$. So it’s a group, but its elements are automorphisms of $S_3$,
$$\operatorname{Aut} S_3 = \{σ \colon S_3 → S_3;~σ~\text{is a group automorphism} \}.$$
Keep in mind that automorphisms are meant to be maps from a group to itself – and not to another group. It stems from the greek word αὐτóς, meaning “self” or “same”, so its something that shapes (grk. “μορφóω”) the group into itself, the same group.  Automobiles are things that movable (lat. “mobilis”) by themselves. Most terms in mathematics actually have meaning.
A: I think   this

In class we've been thinking about automorphisms as a function that
maps elements of one group to another.

is not quite right. Homomorphisms are functions that map the elements of one group to another (and preserve the group multiplication).
When the domain and codomain of a homomorphism are the same group and the homomorphism is bijective (so an isomorphism) then you call that homomorphism an automorphism.
If you have a group $G$ (say, $S_3$) then you can consider all the automorphisms of that group. The set of those automorphisms is itself a group (under functional composition) called the automorphism group of $G$.
You have been asked to find that group - start by listing its elements.
.
