How far can we go with group isomorphisms? The following is quoted from Wikipedia:

From the standpoint of group theory, isomorphic groups have the same properties and need not be distinguished.

But this statement is too general. What I'm wondering about is whether there is a limit to these properties for which isomorphic groups "need not be distinguished".
For instance, I have been thinking about this example: Let $H_1,H_2\lhd G$ and $ S_1,S_2\lhd T$, and $H_1\cong S_1$, $H_2\cong S_2$, I think it is generally NOT true that $$H_1H_2\cong S_1S_2$$
unless some other conditions (like comutativity between subgroups, and that the two subgroup isomorphisms are the same) are imposed on these subgroups. So perhaps even isomorphism can sometimes fail?
If so, then under what circumstances will isomorphism fail (in the sense that even isomorphic groups are different in some way)? Or, in other words, how can we properly make use of an isomorphism condition, without abuse?
 A: $\newcommand{\ZZ}{\mathbb{Z}}$
The groups may be isomorphic, but when you write something like $H_1 H_2$, you're omitting some information. To talk about this object, you need to know about their parent group $G$, and how $H_1$ and $H_2$ embed into it. And if the way they do so is different from how $S_1$ and $S_2$ embed into $T$, then you'll get a different group.
Here's an (admittedly simple) example. Let $G = T = \ZZ_2 \times \ZZ_2 = \{ e, a, b, ab \}$. Let $H_1 = H_2 = S_1 = \{ e, a \}$ and $S_2 = \{ e, b \}$. Clearly, $H_1 \cong S_1$ and $H_2 \cong S_2$. But $H_1 H_2 = H_1$ and $S_1 S_2 = G$, and these aren't even the same size, much less isomorphic.

This can fail with certain other group properties as well. As a (pretty trivial) example, think about the example above, and $H_1 \cap H_2$ vs $S_1 \cap S_2$. Another property that fails is quotient groups. I think products of groups will always work out though.

One way of thinking of this is that $H_1$ and $H_2$ aren't subgroups of $G$; they're just separate groups that come with an inclusion map, $\iota_i : H_i \to G$. This map is injective, and just sends an element in $H_i$ to its 'copy' in $G$. These maps characterize how $H_i$ is contained in $G$. So if you want to show $H_1 H_2$ and $S_1 S_2$ are isomorphic, you need to find isomorphisms $\alpha: H_1 \to S_1$, $\beta : H_2 \to S_2$, and $\gamma : G \to T$ such that these inclusion maps can "come along for the ride".
As a picture (commutative diagram), you need to find $\alpha$, $\beta$, and $\gamma$ such that it doesn't matter how you follow the arrows.
$$
\require{AMScd}
\begin{CD}
H_1 @>{\iota_1}>> G @<{\iota_2}<< H_2 \\
@V{\alpha}VV @V{\gamma}VV @V{\beta}VV \\
S_1 @>{\iota_3}>> T @<{\iota_4}<< S_2
\end{CD}
$$
With the example above, let all three isomorphisms just be the equality map. Then this diagram does not have the desired property. Here's the problematic area: look at $a \in H_2$. If you follow $\iota_2$ to $G$, you get $a$. Following $\gamma$ will give you $a \in T$. But if you follow $\beta$ first, you get $b \in S_2$, and then $\iota_4$ will send you to $b \in T$. We say the diagram "fails to commute".
A: You need to use the correct notion of isomorphism here: namely, you need to choose an isomorphism between $G$ and $T$ which sends $H_i$ to $S_i$. Such an isomorphism need not exist given only that $H_i$ and $S_i$ are abstractly isomorphic. 
In other words, the problem here is that $H_1 H_2$ is not a "property of a group"; it depends on three things, namely a group $G$ and two subgroups $H_1, H_2$ of it, and the correct notion of isomorphism for this situation depends on all three of these pieces of information. 
A: It helps to think of isomorphisms not only as an equivalence of groups, but (in the same (mental) breath) also as a bijective function between the groups.  So, when you say $H_1 \cong S_1$, you should also imagine $f_1:H_1 \rightarrow S_1$, a bijective group homomorphism.  Similarly for $f_2:H_2 \rightarrow S_2$.  With those maps, you can ask the question, "Does $f_1(H_1 \cap H_2) = S_1 \cap S_2 = f_2(H_1 \cap H_2)$ and, more strongly, $\left. f_1 \right|_{H_1 \cap H_2} = \left. f_2 \right|_{H_1 \cap H_2}$?"  A positive answer to this question tells you that $f_1$ and $f_2$ agree everywhere they are simultaneously defined.  Subsequently, the (inner, set) products you mention will be isomorphic.  A negative answer tells you that $f_1$ and $f_2$ do not agree on their common domain, so the products you mention will not be isomorphic.
Note, however, that if we change to an "outer" setting, i.e., we dispense with $G_1$ and $G_2$ and start with four disjoint groups $H_1, H_2, S_1, S_2$, then the set products you write don't make any sense.  (For $h_1 \in H_1$ and $h_2 \in H_2$, $h_1 h_2$ isn't an element of any set.  We would go through the construction of the outer direct product to make sense of this product of elements.)  This is some of the evidence that in addition to the isomorphisms you mention, you will also need some compatible map between the $G$s for the isomorphisms on the factors to lead to isomorphism of the products.
