Trying to understand the differences between $\mathbb{Z}_2 * \mathbb{Z}_2$ vs $\mathbb{Z}_2 \times \mathbb{Z}_2$ I’m trying to understand the differences between free products and direct products with an example: $\mathbb{Z}_2 * \mathbb{Z}_2$ vs $\mathbb{Z}_2 \times \mathbb{Z}_2$. If I understand correctly, the latter should be finite whereas the former is not, is that correct?
 A: Yes, that's correct.
One way to view $\Bbb Z_2\ast\Bbb Z_2$ is via the presentation
$$\langle a,b\mid a^2,b^2 \rangle;$$
on the other hand, $\Bbb Z_2\times\Bbb Z_2$ is given by
$$\langle x,y\mid x^2, y^2,  
 xy=yx\rangle.$$
From this, it can be seen that $\Bbb Z_2\times\Bbb Z_2$ is abelian, while $\Bbb Z_2\ast\Bbb Z_2$ is not.
A: The group $G \times H$ is defined by the set of ordered pairs $(g, h)$ where $g \in G$ and $h \in H$.
On the other hand, $G * H$ can be thought of as the set of lists which alternate between elements of $G$ and elements of $H$:
$$[g_1, h_1, g_2, h_2, \ldots, g_n, h_n]$$
where all elements are not the identity, with the exception of $g_1$ and $h_n$. (I.e. $g_i \ne \varepsilon$ for $i \ge 2$ and $h_i \ne \varepsilon$ for $i \le n-1$.)
Therefore you are correct, $G \times H$ is finite if $G$ and $H$ are finite, while $G * H$ contains infinitely many elements because it contains lists of arbitrarily large length.
Another observation: based on the above, we can see that $G \times H$ maps into $G * H$ via the set map $(g, h) \to [g, h]$. However, while this map is an injection it is not a homomorphism, since we can't nicely multiply elements of the form $[g, h]$ in $G * H$. However, $G \times H$ can be obtained as a quotient group of $G * H$.
A: In addition to the explicit constructions given in the other answers, I think it is worth explaining the difference in characterization of the two objects, the product $G\times H$ and the free-product $G*H$. (Arguably, the names don't help much.)
The product $G\times H$ is a group with "projections" $p:G\times H\to G$ and $q:G\times H\to H$ such that, for any other group $X$ and homomorphisms $p':X\to G$ and $q':X\to H$, there is a unique $\varphi:X\to G\times H$ such that $p'=p\circ \varphi$ and $q'=q\circ \varphi$. Drawing a diagram is a good thing to do here. (Yes, the usual construction produces such an object. There is no need to assume that the underlying set is the Cartesian product.) The characterization proves that such a thing (if it exists at all), is unique up to unique isomorphism, which is a very good thing.
The analogous characterization of free product $G*H$ "has the arrows reversed", and therefore is often called a coproduct, in analogy with other categorical stories. That is, $G*H$ is a group with homomorphisms $i:G\to G*H$ and $j:H\to G*H$ such that, for any other pair $i':G\to X$ and $j':H\to X$ to another group, there is a unique homomorphisms $\varphi:G*H\to X$ such that $i'=\varphi\circ i$ and $j'=\varphi\circ j$.
