What is the purpose of subtracting the mean from data when standardizing? What is the purpose of subtracting the mean from data when standardizing? 
and What is the purpose of dividing by the standard deviation?
 A: Another reason is accuracy.
When computing the variance,
if the mean is large,
much accuracy can be lost.
For example,
the formula for the variance is
$\dfrac{1}{n} \sum_{i=1}^n (x_i-\bar x)^2
$
(you can write $\dfrac1{n-1}$ 
instead of $\dfrac1{n}$
if it makes you feel better).
If the $x_i$ are all close,
even if their mean is large,
this will be quite small.
If you write this in the
mathematically equivalent form
$\left(\dfrac{1}{n} \sum_{i=1}^n x_i^2\right)
-\left(\dfrac{1}{n} \sum_{i=1}^n x_i \right)^2
$,
you will be subtracting
two large quantities
to get a small quantity.
This is the standard recipe
for catastrophic cancellation
and loss of accuracy.
By the way,
if you do a Google search for
"online mean and variance",
you get a number of useful links
including this one from Wikipedia:
http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance.
A: Think of temperature measurements. The numerical values of mean temperature depends on whether we use Fahrenheit or Celsius scale, or some other. It's subject to our arbitrary choice of zero mark on the scale. By subtracting the mean, we remove the influence of that choice. But the choice of unit is still visible in the data because the notion of "$1$ degree change of temperature" is different on different scales. Division by $\sigma$ removes the units: we get a unitless quantity ("$z$-score") which is independent of the temperature scale used. (Well, as long as the scale is linear and warmer means higher temperature.) Now it makes sense to compare our data to some standard distribution such as  $f(x)=\frac{1}{2\pi}\exp(-x^2/2)$ (which is a unitless quantity). 
Shorter version:  the purpose of subtracting the mean from data  when standardizing is to standardize. 
Also, what copper.hat said in comments.
