Why does exponentiating group elements by integers always make sense? Let $G$ be a group.
For any $g\in G$, we can define a mapping $\mathbb Z\to G$ via $n\mapsto g^n$.
This map is a homomorphism between the additive group $\mathbb Z$ and $G$.
On the one hand, it's obvious why we can always do this. We just define $g^n$ as "multiply $g$ with itself $n$ times" (for $n>0$, and then straightforwardly generalise to all $n\in\mathbb Z$).
Thus, $\mathbb Z$ comes into play because we are "counting" the number of times $g$ is operating on itself.
On the other hand, this means that for any group $g\in G$, there is a homomorphism $\mathbb Z\to G$ sending $n$ to $g^n$.
Is there any reason why this group in particular, $\mathbb Z\equiv(\mathbb Z,+)$, ends up being homomorphic to subgroups of every group? Are there groups other than $\mathbb Z$ sharing this property?
 A: First, for each $g$ your map is really a homomorphism of the integers $\mathbb{Z}$ to $G$. You map $-1$ to $g^{-1}$. You don't want to restrict the map to the positive integers.
All you are really saying is that any group has lots of cyclic subgroups - every element generates one. Any any cyclic group is a homomorphic image of $\mathbb{Z}$.
You can always map any group $H$ homomorphically to $G$ with the trivial map the sends everything to the identity.
A: Because the group is equipped with an operator which for which this set has closure. 
Anything that the operator results will remain in the set as long as the inputs are in the set.
So we can prove using induction : 


*

*Firstly, observe $g^0 = e \in G$ (base case), must exist such an $e$ (identity element)  in every group.

*Secondly because of the closure property, for any $a \in G$, we have:   $$g^{n}a = g^{n-1}\circ \underset{b\in G}{\underbrace{g \circ a}}$$
So for any $n$ this property exists iff it exists for $n-1$, and it does exist for $0$. 
Therefore by induction it must exist for all $n$.
A: As you explain, given any group $G$ and any element $g\in G$, there is a (unique) homomorphism $\mathbb Z\to G$ which takes $1$ to $g$. You ask whether other groups have similar properties. As mentioned in a comment on another answer, the relevant concept is that of a free group. The infinite cyclic group $\mathbb Z$ is a free group on one generator. Now, for any set $S$ there is a free group over $S$, i.e. a free group $F$ with generators (identified with) the elements of $S$, and any (set) map from $S$ to $G$ can be uniquely extended to a homomorphism $F\to G$.
A: Just as there exists the embedding that maps trivial group $C_0$ to the identity of any group $H$, for all groups $G$, you can always maps it to the trivial subgroup of $H$.
Some phenomena from category theory seems related the universal property you described. For example, the functor $U:\mathbf{Grp}\to\mathbf{Set}$ that maps every group to its set of elements is representable, since $U\cong \mathrm{Hom}_{\mathbf{Grp}}(\mathbb{Z}, -)$. On the other hand, $\mathbb{Z}$ as a ring (with 1) has the property of being “initial”: it is the only ring such that for all rings $R$, there exists a unique homomorphism $\mathbb{Z}\to R$.
