What is this notation in the definition of an exterior derivative? What does this notation inside the red boxes mean?  Is it a product of $a$'s and $\alpha$'s, or is it just a single element in it's respective set with the comma separated indexes to specify it's relation to each $e_i$?

 A: Each $a_{i_1,\dots,i_p}$ is just a function, the coefficient of $e^{i_1} \wedge \dots \wedge e^{i_p}$, hence the notation. No products understood.
A: No, it's not a product. It is a single member of $A$, indexed by a multi-index. Although the subscript is written without commas in the cited page, you could certainly think of it with commas separating the subscripts. Since a basis for the module of tensors or wedges carry multiple basis elements for $M$, we need multiple indexes. 
For example, if $M$ has basis $\{e_1,e_2,e_3\}$, then a basis for $\Lambda^2 M$ is $\{e_1\wedge e_2,e_2\wedge e_3,e_1\wedge e_3\}.$ So an arbitrary element $a\in\Lambda^2 M$ may be written $a= a_{12}\,e_1\wedge e_2+a_{23}\,e_2\wedge e_3+a_{13}\,e_1\wedge e_3.$ So $a_{12}$ just means the coefficient of the basis element depending on $e_1$ and $e_2$.
A: You can think of $a_{i_1 \cdots i_p}$ as instead $a_{i_1, \cdots, i_p}$ (where I've used commas), or as $a(i_1, \cdots, i_p)$ (where I'm expressing $a$ as a function of the indices). But the comma-omitted version is perhaps most popular. It's reminiscent of the iterated derivative notations $f_{xy}$ or $D_{xy}$ often used to refer to two consecutive partial derivatives (and not a single derivative with respect to $xy$ or something like that).
