What makes the Cauchy principal value the "correct" value for a integral? I haven't been able to find a good answer to this searching around online. There is a related old question here, but it never received much attention.
Suppose I have some physical property that I believe depends on $\int_{-\infty}^{\infty}xdx$. Ordinarily, we would say the integral is undefined, but we could also hypothetically take the Cauchy principal value and say the integral evaluates to zero. 
My question is under what conditions should I let the integral remain undefined and when should it equal zero? It seems odd to me that based on different circumstances of the problem we could say that the same integral has different values. For pure math, as long as you work within a consistent framework, it doesn't really matter which you decide is true. But for physics, chemistry, etc if the integral relates to some property, it would seem there would be some definitive, empirical solution to the problem. 
This question arises from some problems I have been doing with the Cauchy distribution. It seems that for certain physical examples, there is certain camp that treats the location parameter as the mean of the distribution, which is effectively saying the integral to determine the mean is equal to its principal value. I don't like this because it seems to suggest to me that all the higher order odd moments could also be argued to exist, at least in case where the distribution is symmetric about zero, 
 A: I like Abhimanyu Pallavi Sudhir answer: we can assign the integral $\int_{-1}^1\tfrac1x\,dx$ the value $\infty$, $-\infty$ or any other number, and we would be equally right. There is no criterion to select a result over the other. Sometimes this is presented as a flaw of the Lebesgue integral, which I think is nonsense; this integral just does not have an unambiguous meaning and it is not the fault of Lebesgue or of Riemann.
If, however, we consider the integral operator
$$
T(f):=\int_{-1}^1 \frac{f(x)}{x}\, dx, $$ 
where $f$ is an arbitrary smooth function, then things are different. Let us develop $f$ in a Taylor series; we get, formally,
$$
\int_{-1}^1 \frac{f(x)}{x}\, dx = f(0)\int_{-1}^1\frac{dx}{x} + f'(0)\int_{-1}^1\, dx + \ldots$$ 
The only ambiguous term is the zeroeth-order one; all the others make perfectly sense as standard integrals. 

Conclusion. We need a renormalization procedure that removes the zeroeth-order term. This procedure amounts to exploit symmetry, prescribing that $\int_{-1}^1 \frac{1}{x}\, dx$ should be interpreted as $$ \lim_{\epsilon \to 0}\int_{\epsilon<\lvert x \rvert <1} \frac{dx}{x}=0,$$ that is, considering integrals in the principal value sense.

This is the starting point of the Calderón-Zygmund theory of singular integrals. In general, given 
$$
K(x)=\frac{\Omega(\tfrac{x}{|x|})}{|x|^n}, $$ 
where $\Omega$ is a function on the unit sphere $\mathbb S^{n-1}$ with zero integral, that is, 
$$
\int_{\mathbb S^{n-1}}\Omega\, dS =0, $$ 
we can define an integral operator 
$$
Sf(x):=\int_{\mathbb R^n} K(x-y)f(y)\, dy, $$ 
where the integral is interpreted in the principal value sense, exactly as before. Some conditions must be put on $K$ to ensure that this operator is continuous in appropriate function spaces. But most importantly, this definition based on the principal value is not crazy, or arbitrary, as it might seem at first sight. It is the only possible choice that removes the singular term from the integral, while leaving all the other terms untouched.

A final remark. I think that "principal value" is a gallicism, which obscures the true meaning. "Renormalization" would be better, as Gerry Folland suggests in his book "Real Analysis". 
A: It isn't the "correct value" for the integral any more than the principal root is the correct value for a root or the principal logarithm is the correct value for a logarithm, or setting $C=0$ gives you the correct value of the antiderivative. There are plenty of other values the integral can take, depending on how you take the limit. See my answer to Why can't $\int_{-1}^1\frac{dx}x$ be evaluated?
A: A physics related example may also be useful. Consider a potential of the form $V=-Log(\left|x\right|)$. Then the force is given by
$$
F=-{d V \over d x}={1\over x}.
$$
The work done on a particle which moves from $-a$ to $a$ (where $a>0$) is
$$
W=\int_{-a}^a{1\over x} dx.
$$
From conservation of energy, we would expect $W=0$ as $V(a)=V(-a)$ and so, in that case, the Cauchy principal value gives the correct answer. However, in general, we would expect that it is not physically possible to have an infinite potential and so some new physics would come in near to $x=0$ which would change the shape of the potential in that region. But, provided that energy is still conserved we would still get $W=0$ when moving the particle from $-a$ to $a$ and so the Cauchy principle value would still have given the correct answer even without accounting directly for the new physics.
Another interesting example is when $V=-\frac{1}{x}$ which is plotted below:
If you imagine a ball being rolled from $x<0$ it would need infinite kinetic energy to get to $x>0$. However, we expect some new physics to come in and keep $V$ finite and continuous around $V=0$. In that case the difference in potential energy between $x=-1$ and $x=1$ would simply be $V(-1)-V(1)=2$. So we would want
$$
W=-\int_{-1}^1 \frac{1}{x^2}\,dx=2
$$
As explained in this answer, for this case the Cauchy principle value for the integral gives $-\infty$ while contour integration gives $2$. So in this case, contour integration is better than Cauchy's principle value from a physics perspective.
A: The Cauchy principal value is very important, especially in cases where the Lebesgue integral (which it seems you refer to as the improper integral) does not exist. The issue is that the Lebesgue integral doesn't deal too well with really big oscillations. Indeed, a measurable function $f$ is Lebesgue integrable if and only if $|f|$ is, so oscillations don't matter - only the magnitude does. This becomes problematic when dealing with functions such as $\frac{\sin x}{x}$ on $(0,\infty)$ or $\frac{1}{x}$ on $(-1,1)\setminus\{0\}$.
A purpose of the Cauchy principal value is to rectify this problem, to take into account oscillations like the Riemann integral does and give a meaningful number that represents the integral (i.e. scaled average) of the function in question. The Cauchy p.v. of $\frac{1}{x}$ is $\lim\limits_{\epsilon \to 0^+} \int_{-1}^{-\epsilon} \frac{1}{x}dx+\int_{\epsilon}^1 \frac{1}{x}dx = 0$, which coincide with our intuition for what the average value of $\frac{1}{x}$ should be.
The most prominent use of the Cauchy principal value is the Hilbert transform, in which we study $\int \frac{f(x-y)}{y}dy$, which of course, needs to be defined properly. It is critical here that we don't just say the improper integral exists, but rather get a quantitative sense of the oscillation present.
