The intuition behind Trig substitutions in calculus I'm going through the MIT open calculus course, and in one of the lectures (19-28min marks) the professor uses the trig substitution $x = \tan \theta$ to find the integral of $\frac{dx}{x^2 \sqrt{1+x^2}}$.
His answer: $-\csc(\arctan  x) + c$, which he shows is equivalent to $-\frac{1+x^2}{x} + c$ by drawing a right triangle on the blackboard.
I get the math behind each step of it, but I can't wrap my head around why that equivalence works.  We just used an arbitrary $x = \tan \theta$ substitution, where $\theta$ moves differently than x does, and the expression $-\frac{1+x^2}{x} + c$ by itself doesn't know anything about trigonometry.  But I type both into Excel for a bunch of different x values, and obviously they are equivalent.
I guess I'm not really sure what my question is here, but I could just use some perspective.  It just seems like substituting ANY function in for x then integrating it shouldn't work, especially when crossing into polar coordinates.
 A: There are two key things to consider in this example:
A good way to try to integrate a function with a square root of a quadratic is to make a trig substitution, since the appropriate trig substitution will get rid of the square root.  In particular, when you have a square root of the form $\sqrt{x^2+a^2}$, substituting $x=a\tan\theta$ is a good thing to try since then the square root becomes $a\sec\theta$.
Whenever you have an expression of the form $f(g(x))$ where $f$ is a trig function and $g$ is an inverse trig function, then it can be simplified using a right triangle or using trig identities.
A: The basic "family" of trigonometric substitutions uses the definitions of trig functions to produce a ratio $ \ \frac{x}{a} = \ $ [trig function] $\theta \ \rightarrow \ x = a \cdot $ [trig function] $\theta \ $ , $ \ a \ $ being a constant.  The legs and hypotenuse of a right triangle can produce three possibilities relating square roots of the sum or difference of two squared terms:
$ \sqrt{x^2 + a^2} \ $ can only be the hypotenuse of a right triangle with legs of lengths $ \ x \ $ and $ \ a \ $ .  It is the usual practice to make $ \ x \ $ the leg opposite the angle $ \ \theta \ $  and $ \ a \ $ the "adjacent" leg, which produces the ratio $ \ \tan \theta = \frac{x}{a} \ $ , as described above.  The associated differential is then $ \ dx = a \sec^2 \theta \ d\theta \ $ ; the radical is then given by $ \frac{\sqrt{x^2 + a^2}}{a} = \sec \theta \ \rightarrow \ \sqrt{x^2 + a^2} = a \sec \theta \ $ .
The other arrangements involve differences of squared terms, which requires the radical to be the length of one of the triangle's legs.  We have either
$ \sqrt{x^2 - a^2} \ $ , which makes $ \ x \ $ the hypotenuse; generally, the side adjacent to $ \theta \ $ is chosen as the leg $ \ a \ $ , leading us to $ \ \sec \theta = \frac{x}{a} \ \rightarrow \ dx = a \sec \theta \tan \theta \ d\theta \ $ and $ \frac{\sqrt{x^2 + a^2}}{a} = \tan \theta \ \rightarrow \ \sqrt{x^2 + a^2} = a \tan \theta \ $ 
or
$ \sqrt{a^2 - x^2} \ $ , making $ \ a \ $ the hypotenuse; the side opposite to $ \theta \ $ is usually chosen to be the leg $ \ x \ $ , giving $ \ \sin \theta = \frac{x}{a} \ \rightarrow \ dx = a \cos \theta \ d\theta \ $ and $ \frac{\sqrt{x^2 + a^2}}{a} = \cos \theta \ \rightarrow \ \sqrt{x^2 + a^2} = a \cos \theta \ $ . 
$$\\$$
Returning to your integral, the "tangent substitution" (with $ \ a \ = 1 \ $ ) produces
$$\int \ \frac{1 \cdot \sec^2 \theta \ d\theta}{1^2 \cdot \tan^2 \theta \ \cdot \ 1 \cdot \sec \theta} \  = \ \int \ \frac{\cos \theta \ d\theta}{ \sin^2 \theta } \ , $$
which can now be completed through a "$v-$subsitution" , $ \ v = \sin \theta \ $ , yielding the result 
$$\int \ v^{-2} \ dv \ = \ -v^{-1} \ + \ C \ \rightarrow \ -\frac{1}{\sin \theta} \ + \ C \ \ \text{or} \ \ -\csc \theta \ + \ C \ . $$ 
[Maybe this was already clear to you, but you also seemed to be asking about the rationale for the choice of particular trig functions...]
Referring back to the associated right triangle, the side opposite $ \ \theta \ $ is $ \ x \ $ and the hypotenuse is $ \sqrt{x^2 + 1^2} \ $ , so the "back substitution" to return to a function of $ \ x \ $ gives us $ \ -\frac{\sqrt{x^2 + 1}}{x} \ + \ C \ $ . (I believe you have omitted a radical in your expression; a check against WolframAlpha confirms this...)
Were we to simply construct a right triangle with the legs and hypotenuse described (and thus $ \ \theta = \arctan \frac{x}{1} \ $ ) and ask for the expression representing $ \ -\csc \theta \ $ , we would obtain this same result.
A: This has to do with the fact that although $1/\sqrt{1+x^2}$ is difficult to integrate,
$$1/\sqrt{1+\tan^2 t}= 1/\sqrt{1/\cos^2 t}=\cos t$$
And together with $dx = dt/\cos^2 t $, leads to a very simple integral. In essence, any substitution "works", in the sense that all lead to the same numerical value, but only some of these lead to an easier integration.
As to the general idea behind substitution, take the easy example of $t = ax$:
$$\int_0^1 f(x) dx = \int_0^a \frac{f(x/a)}{a} dt $$
What's happening here? If you think geometrically, we just stretched the shape along the $x$-axis, so it's width is now $a$ instead of $1$. If we were to just naively substitute $t=ax$, the area would be $a$ times larger! and that is why we need to correct for this "stretching" by dividing by $dx/dt=a$.
As you mentioned, in the more general case the stretching is no longer uniform, but you can look at the new integral as a sum of differently stretched pieces, each with it's own correction factor $$dt/dx \neq \rm{const}$$.
By way of example, take a look at the integral over the half circle $\sqrt{1-x^2}$, with the transformation $x=\sin(t)$. You can see how the uniformly distributed widths in $x$ get "squeezed" differently in $t$:

A: Your method is based on the identity
$$
1+\tan ^{2}\theta =\sec ^{2}\theta .
$$
But there is another standard method to integrate  by substitution an irrational 
function of the type $f(R(x),\sqrt{a^2+x^{2}})$, where $R(x)$ is a rational function of $x$. This alternative method, which is based on the identity
$$1+\sinh ^{2}t=\cosh ^{2}t,$$
uses the hyperbolic substitution 
$$
x=\sinh t\Rightarrow dx=\cosh t\,dt,\qquad \text {or}\qquad x=a\sinh t,\quad \text {for a given } a.$$
As a matter of fact
\begin{eqnarray*}
\int \frac{dx}{x^{2}\sqrt{1+x^{2}}} &=&\int \frac{\cosh t}{\sinh ^{2}t\cosh t
}\,dt,\qquad x=\sinh t \\
&=&\int \frac{dt}{\sinh ^{2}t}=-\frac{\cosh t}{\sinh t}+C \\
&=&-\frac{\sqrt{1+x^{2}}}{x}+C.
\end{eqnarray*}
The key issue is that both the trigonometric and the hyperbolic
substitutions lead to simpler integrals, because the irrational integrand becomes a rational function of trigonometric or hyperbolic functions. 
