Why do we even need transitivity? I am referring to two Peano axioms:
If $x = y$ then $y = x$ (symmetry)
If $x = y$ and $y = z$ then $x = z$ (transitivity)
What I don't understand is why we need the transitive axiom. Isn't it already implied by the symmetry axiom? Is it more of a convenience? Can we prove that transitivity is a required axiom? What happens if we were to remove it?
Or should I think of this more like "symmetry just says we can physically flip the order of the equalities, and transitivity lets us swap things around as long as they're all equal to each other"?
 A: It's easy to write down an example of a symmetric relation which fails to satisfy transitivity: on the set $\{a,b,c\}$ we say that $a \sim b$ and $b \sim a$ and $b \sim c$ and $c \sim b$ (I'm using a different symbol $\sim$ so as not to confuse the issue). So no, the symmetry axiom does not imply the transitivity axiom. 
Perhaps you might want to think of the transitive law as it is stated in some translations of Euclid: thing which are equal to the same thing are equal to each other.
A: Consider the following relation: $x$ opposes $y$ if and only if $x = -y$.
Can you see that the "opposes" relation is symmetrical but not transitive?
A: Some relations are symmetric without being transitive.
For  example if John and Jill are friends and Jill and Jeff are friends, it does not imply that John and Jeff are friends. 
So friendship is a relation which is symmetric but not transitive. 
Mathematically speaking, in geometry perpendicularity is symmetric but not transitive.
We like the relation $x=y$ be both symmetric and transitive.   
A: Another example of a relationship which is symmetric but not transitive: computer floating point arithmetic is approximate, so for example (1./3.)*3. != 1.0 (some rounding occurs when the division is done, and this destroys the equality).  It is thus important when comparing floating point results for equality to compare with an appropriate margin of error.  Deciding what this margin should be is a hard problem, but let us use a margin of 0.1:
Then 0.61 is near 0.7 and 0.7 is near 0.61 (symmetric), but whereas 0.7 is near 0.79, 0.61 is not near 0.79 (not transitive).
A: Other answers cover very well why we need a transitivity axiom in the case of axiomatised Peano arithmetic.
But it's worth considering: why does this not feel necessary for the equality relation? The following idea may underly your intuition.
In many logics, we have an additional rule of 'substitution' or 'substitutivity'. Sometimes it's considered an axiom, sometimes a rule part of the logic, and sometimes an axiom schema (for example sometimes in First Order Logic, in which the second order axiom given below can not be written). Read about it here on Wikipedia.
It looks something like this:
For all variables $x$ and $y$ and for all formulae $P$, if we know $x = y$ and we know $P(x)$ then we know $P(y)$.
One way to write it as a second order statement is this:
$\forall P. \forall x. \forall y. x = y \implies P(x) \implies P(y)$.
Interestingly, once we have this axiom, logical rule, or axiom schema in place, all that is required to derive the other properties of equality is Reflexivity.
The derivation of Symmetry relies on the observation that an application of the substitution rule with $P$ as $w \mapsto w = x$ gives us
$\forall x. \forall y. x = y \implies x = x \implies y = x$.
The derivation of Transitivity relies on observation that an application of the substitution rule with $P$ as $w \mapsto w = z$ gives us
$\forall x. \forall y. x = y \implies x = z \implies y = z$.
