Coset operations Let $S$ be a subset (not necessarily a subgroup) of a group $(G,\cdot)$. For any $g\in G$, the coset $gS$ is defined by
$$gS = \{g\cdot s| s\in S\}\,.$$
Let $C(S)$ be the collection of all cosets of $S$. If the binary operation $\times :C(S)\times C(S) \to C(S)$ defined by
$$
aS\times bS = (a\cdot b)S
$$
is well-defined, does $S$ need to be sub-group of $G$?
Answer: If $S$ is a singleton subset, it is obvious that the operation is well-defined and yet $S$ may not be a subgroup. 
Question: What if $S$ has more than one element in it?
 A: Take $S$ to be some coset of a normal subgroup that isn't the subgroup itself.  Then the multiplication is well-defined and yet $S$ is not a subgroup.  Perhaps a better question is "If the multiplication is well-defined, is some coset of $S$ a subgroup of $G$?"
There are difficulties with cosets of arbitrary subsets.  The main one seems to be that $C(S)$ (usually written $G/S$ but I'll stick with your notation) is not necessarily a partition of $G$.  For instance, the cyclic group $C_n=\langle x\mid x^n\rangle$ with $S=\{1,x\}$.  Another is that the stabilizer of $S$ (i.e., $\{g\in G:gS=S\}$) when $S$ is a subgroup is $S$ itself, but when $S$ is not a subgroup it is less clear how to proceed.
Suppose $S$ is a subgroup such that $(aS)\cdot (bS):=abS$ is well-defined.  Without loss of generality, assume $S$ contains the identity (if $x\in S$, replace $S$ with $x^{-1}S$; this does not change $C(S)$).  If $xS=S$, then  $x\in S$ since $S$ contains the identity, hence $S$ contains the stabilizer of $S$. (Conjecture: if the stabilizer is a normal subgroup, then the multiplication is well-defined.)
Take $G=C_3$ with $S=\{1,x\}$.  The stabilizer of $S$ is the trivial subgroup, so the cosets are $S,xS,x^2S$.  The multiplication is well-defined, but $S$ is not a coset of a subgroup of $G$.
