Definition of standard deviation and $l_2$ If we denote the mean as $\mu$, then the standard deviation is:
$$\sigma\equiv\left(\sum_{x\in X}{p(x)(x-\mu)^2}\right)^\frac{1}{2}$$
In other words, $\sigma$ is the average $l_2$ distance from $\mu$.
In general, if we use a different metric
$$\sigma_n \equiv \left(\sum_{x\in X}{p(x)|x-\mu|^n}\right)^\frac{1}{n}$$
It seems that the choice of $n=2$ is quite arbitrary, Is there a reason for that?
 A: No, $\sigma$ is not the average $\ell^2$ distance from $\mu$.  The $\ell^2$ distance from $x$ to $\mu$ is $\sqrt{(x-\mu)^2} = |x-\mu|$ and the average of those is
$$
\sum_{x\in X} p(x)|x-\mu|.
$$
Rather $\sigma$ is the $\ell^2$ distance from the tuple of $x$ values to the tuple in which every component is $\mu$.
A reason for the use of the mean squared deviation is that is it additive.  Suppose $p$ and $q$ are probability mass functions on $X$ and $Y$ and $\mu$ and $\nu$ are averages with respect to those two measures.  Then
$$
\sum_{(x,y)\in X\times Y} ((x+y)-(\mu+\nu))^2p(x)q(y) = \sum_{x\in X} p(x)(x-\mu)^2 + \sum_{y\in Y} q(y) (y-\nu)^2. \tag 1
$$
Thus, suppose I toss a coin $1800$ times.  The expected number of heads is $900$; the standard deviation of the number of heads is${}\,\ldots\,{}$what??  The equality $(1)$, iterated, quickly gives me the answer to that question.  With that answer, I can use the central limit theorem to find the probability that the number of heads is between $890$ and $915$ inclusive.
