Let me expand upon Qiaochu's answer.
It is a fact that $X \times Y$ is a categorical product, meaning that it has a universal property, namely this: it comes equipped with two maps, called the projections, $\pi_1:X \times Y \to X$ and $\pi_2:X \times Y \to Y$ satisfying the following property. For any other object $Z$ and maps $\alpha:Z \to X$ and $\beta:Z \to Y$, there is a unique map $\gamma:Z \to X \times Y$ such that $\pi_1 \circ \gamma = \alpha$ and $\pi_2 \circ \gamma = \beta$.
This ensures that products are unique up to unique isomorphism. This is done, in sketch, as follows: any two products both satisfy the universal property of products, hence there are unique maps to both of them. Composing these unique maps we get a unique map from one of the proposed products to itself. But the identity map also satisfies this property, and by uniqueness, the composition must be the identity. Hence they are isomorphic.
This is enough to show that products are associative. One does this by showing that two orderings of the parentheses satisfy the universal property, hence they must be isomorphic by the above argument.