Why don't all odd functions integrate to $0$ from $-\infty$ to $\infty$? Why don't all odd functions integrate to $0$ from $-\infty$ to $\infty$?
As for any odd function $f(x)$, 
$$\int_{-a}^{a} f(x) dx = 0$$
I actually ran into trouble in a recent examination where I was asked to compute the mean and standard deviation of Cauchy Distribution and I computed the mean as $0$ (the integrand is odd as the pdf of the distribution is even) and the standard deviation as $\infty$ (because $E(X^2) = \infty$). But the official answer stated that 

Mean is undefined: no unique solution.  

And naturally if mean is undefined, so is the variance as $$Var(X) = E(X^2) - (E(X))^2$$
But I was under the impression that integration of any odd function from $-a$ to $a$ is $0$.
 A: The critical thing is that $\infty$ is not a number, so $\int_{-\infty}^\infty f(x)dx$ is shorthand for $\lim_{a \to -\infty} \lim_{b \to \infty} \int_a^b f(x)dx$.  You can't apply the odd function rule because you are not guaranteed that $a$ and $b$ approach $\pm \infty$ at the same rate.  We normally demand that the integrals converge separately at the positive and negative limits.  If so, we can assign a unique value to the integral, irrespective of how the limits are taken.  The Cauchy principal value comes from defining the integral to be  $\lim_{a \to \infty} \int_{-a}^a f(x)dx$.  Then you can use the fact that the integal of an odd function over a symmetric interval is zero to say that the limit is zero.  
To see why the Cauchy principal value is problematic, take $\int_{-\infty}^\infty x\; dx$.  The principal value is zero because it is integrating an odd function.  Let us do a $u$ substitution with $$u=\begin {cases} \frac x2&x \gt 0\\2x& x \le 0 \end {cases}$$  We get $$\int_{-\infty}^\infty x\; dx=\int _{-\infty}^0\frac u4 du + \int_0^\infty 4udu$$ and neither of the integrals converges.  If we try to do a Cauchy principal value here, we get $$\int_{-a}^a f(u)du=-\frac{a^2}8+2a^2$$ which diverges.  It is like summing a conditionally convergent series.  If you rearrange it, you can get any value you want.
A: From a measure theory perspective (which probability is based on):
The main problem comes from the fact that $\infty-\infty$ is not defined: This is what causes many Lebesgue integrals to be undefined.
The integral $$\int_\mathbb{R}f(x)\,dx=\int_{\mathbb{R}}f^+-\int_\mathbb{R}f^-$$
is undefined if both integrals on the right side are infinity, since we run into the $\infty-\infty$ case.
