What exactly does it mean to say "$dv=4\pi r^2\,dr$ can be thought of as the spherical volume element between $r$ and $r+dr$"? In my textbooks and lectures (I'm a second year physics student) I often come across statements such as $$``dV=4\pi r^2dr\text{ can be thought of as the spherical volume element between }r\text{ and }r+dr."$$
Is this meant only symbolically, or is it a precise statement? For instance, I get \begin{align}\int_r^{r+dr}4\pi r^2dr&=\frac{4}{3}\pi\left[r^3\right]_r^{r+dr}\\&=\frac{4}{3}\pi(r^3+dr^3+3rdr^2+3r^2dr-r^3)\\&=\frac{4}{3}\pi(dr^3+3rdr^2+3r^2dr)\\&=4\pi r^2dr+\text{increasingly smaller terms.}\end{align}
Moreover, a Taylor expansion of the spherical volume function at a point $r'$  around a general point $r$ gives $$V(r')\approx\frac{4}{3}\pi r^3+4\pi r^2(r'-r)+\cdots.$$ Since the volume of a spherical shell between $r$ and $r+dr$ is generally (?) the volume of a sphere with radius $r+dr$ minus the volume of a sphere with radius $r$, we get $$\text{volume between }r\text{ and }r+dr\approx\frac{4}{3}\pi r^3+4\pi r^2dr-\frac{4}{3}r^3\approx4\pi r^2dr.$$
This leads me to think that what $4\pi r^2dr$ really is, is a good approximation (in fact, a first order Taylor expansion) of the volume between $r$ and $r+dr$, which can furthermore be assumed to be perfectly valid in the limit $dr\to 0$.
Maybe ... I'm just being pedantic?
 A: You aren't being pedantic: you're just attaching a precise mathematical meaning to the informal statement you quoted.
The same can be said in general. Let $f(x)$ be a differentiable real-valued function of one real variable. 
Then $df=f'(x)dx$ can be thought of as the "differential element of $f$" between $x$ and $x+dx$. (Indeed, in advanced calculus, $df$ is called a "differential," as in "differential form.")
This just means that $$f(x+dx)-f(x)\approx f'(x)dx$$
to first order, as you wrote.
A: You are on right way, not pedantic.
EDIT1:
$$ V=A. h,\quad dV= A\cdot dh$$
A differential volume element $dV$ of a constant  prismatic brick area $A$ and height $h$ (rectangular parallelopiped) from an altitude of $dh$ from a rectangular flat patch of area $A$ which can be length $\times$ breadth.
Now for a sphere, differentiate volume of a sphere w.r.t. radius
$$ V= \frac43 \pi r^3,\quad \frac{dV}{dr}= 4 \pi r^2 =\text{Area}$$
This is a differential volume element $dV$ from a radial growth of $dr$ from a curvilinear or doubly curved patch of area $A,$ the sphere area $4 \pi r^2$. In other words  the volume $V$ grows radially at the rate of its instantaneous surface area $4 \pi r^2:$
$$ dV= A\cdot dr;$$
