Basic question about subgroup generated by a set?

My book first gives the top-down definition of $\langle A\rangle$ (where $A$ is a subset of a group) and then it claims

$$\langle A\rangle = \{a_1^{e_1}a_2^{e_2} \cdots a_n^{e_n}| n \in \mathbb{Z}, n \ge 0 \text{ and } a_i \in A, e_i = \pm 1 \text { for each } i \}$$

This notation is a bit confusing to me; must $a_1, a_2, ... a_n$ be all the elements of $A$? Must the multiplication always be in this fixed order?

The $a_i$'s need not be all the elements of $A$, but just $n$ elements chosen among the elements of $A$. The description in your question specifies that $\langle A\rangle$ must contain all the products $a_1^{e_1}\cdots a_n^{e_n}$, for all $n\geq 0$, all possible ways to choose $n$ (ordered) elements $a_1,\dots,a_n$ in $A$ and all possible ways to choose the $e_i$'s in $\{1,-1\}$.
As for the multiplication order, we choose to write the product in a certain order, but note that in fact all the orders are considered, since they correspond to different choices of $a_1,\dots,a_n$. For example, if $x,y\in A$, then you must have $xy\in \langle A\rangle$ (because choosing $n=2$, $a_1=x$, $a_2=y$ and $e_1=e_2=1$ gives you $a_1^{e_1}a_2^{e_2}=xy$) and also $yx\in \langle A\rangle$ (because choosing $n=2$, $a_1=y$, $a_2=x$ and $e_1=e_2=1$ gives you $a_1^{e_1}a_2^{e_2}=yx$).