What's so "natural" about the base of natural logarithms? There are so many available bases. Why is the strange number $e$ preferred over all else?
Of course one could integrate $\frac{1}x$ and see this. But is there more to the story?
 A: If you consider all exponential equations $a^x$, they all have $y$-intercept $(0,1)$. If you wanted to specify an archetypal exponential equation to refer to as you work through Calculus, a natural choice would be to choose the one whose tangent line at $(0,1)$ has slope 1. The equation $e^x$ is the unique exponential equation with that property.
A: Differentiation and integration is precisely why it is considered natural, but not just because 
$$\displaystyle\int \frac{1}{x} dx=\ln x$$
$e^x$ has the two following nice properties
$$ \frac{d}{dx} e^x=e^x $$
$$ \int e^x dx=e^x+c $$
If we looked at $a^x$ instead, we would get:
$$\frac {d} {dx} a^x= \frac{d}{dx} e^{x\ln(a)}=\ln(a) \cdot a^x$$
$$\int a^x dx= \int e^{x\ln(a)} dx=\frac{a^x}{\ln(a)}+c$$
So $e$ is vital to the integration and differentiation of exponentials.
A: If you know some linear algebra, then here is an abstract reason: $e^x$ is the unique eigenvector of eigenvalue $1$ of the derivative $D$ acting on, say, the space of smooth functions on $\mathbb{R}$.  Why is this important?  The study of solutions of linear differential equations with constant coefficients is equivalent to the study of nullspaces of operators which are polynomials in $D$, e.g. operators of the form $\sum a_k D^k$.  Any such operator automatically commutes with $D$, so this nullspace splits up into eigenspaces of $D$.  That's why solutions to linear differential equations with constant coefficients can be expressed as sums of complex exponentials.  The choice of $e$ makes it particularly easy to see what the eigenvalue is: the eigenvalue of the eigenvector $e^{\lambda x}$ is $\lambda$.
A: The wikipedia article on e tells a bit of the story.

One example is an account that starts with 1.00 and pays 100% interest per year. If the interest is credited once, at the end of the year, the value is 2.00; but if the interest is computed and added twice in the year, the 1 is multiplied by 1.5 twice, yielding 1.00×1.5² = $2.25. Compounding quarterly yields 1.00×1.254 = 2.4414…, and compounding monthly yields 1.00×(1.0833…)12 = 2.613035….
Bernoulli noticed that this sequence approaches a limit (the force of interest) for more and smaller compounding intervals. Compounding weekly yields 2.692597…, while compounding daily yields 2.714567…, just two cents more. Using n as the number of compounding intervals, with interest of 100%/n in each interval, the limit for large n is the number that came to be known as e; with continuous compounding, the account value will reach 2.7182818…. More generally, an account that starts at $1, and yields (1+R) dollars at simple interest, will yield eR dollars with continuous compounding.

Additionally, it is the base of the exponential function y = k^x, finding a specific value for k where d/dx k^x = k^x.  That is, the rate of change of the exponential curve at any point is equal to the y value of the curve at that point.
A: The Wikipedia article about e lists many properties of the constant that make it naturally occurring.
I think the biggest reason it is natural when it comes to exponentiation/logarithms is that it is the only number that satisfies 
$$ \frac{d}{dt} e^t =e^t $$ while every other number satisfies 
$$ \frac{d}{dt} a^t = c \cdot a^t$$ where $c$ is some constant, different than 1.  This makes it "normalized" in a sense.
A: I'm surprised I never answered this; maybe I was deterred by the fact that several other answers are here.
One short answer is this: An exponential function $y=a^x$ grows at a rate proportional to its present size, but only when the base is $e$ does it grow at a rate equal to its present size.  In other words
$$
\frac{d}{dx} a^x = \left(\text{constant}\cdot a^x \right),
$$
but only when $a=e$ is the "constant" equal to $1$.
The number $a=2$ is too small for this to happen.  To see that consider
$$
\frac{d}{dx} 2^x = \lim_{h\to0} \frac{2^{x+h}-2^x}{h} = \lim_{h\to0}2^x\frac{2^h-1}{h}
$$
This last limit is equal to $\displaystyle 2^x \lim_{h\to0}\frac{2^h-1}{h}$.  That step can be done because $2^x$ is a "constant", but in this instance, "constant" means "not depending on $h$".  Then observe that $\displaystyle\lim_{h\to0}\frac{2^h-1}{h}$ is a "constant", where "constant" now means "not depending on $x$".
So
$$
\frac{d}{dx} 2^x = \left(\text{constant}\cdot 2^x\right).
$$
But what number is this "constant"?  Notice that as $x$ increases from $0$ to $1$, $2^x$ increases from $1$ to $2$, so the average slope on that interval is $\dfrac{2-1}{1-0}=1$.  Since the curve gets steeper as $x$ increases, it's not yet that steep at $x=0$.  Its slope at $x=0$ is $\left.\dfrac{d}{dx}2^x\right|_{x=0}=\left(\text{constant}\cdot2^0\right)$, so that "constant" must be less than $1$.
A similar argument shows that if $4$ is used as the base, the "constant" is more than $1$.  This is done by using the interval from $-1/2$ to $0$ instead of the interval from $0$ to $1$.
So $2$ is too small, and $4$ is too big, to be the natural base. $e$ must be somewhere between $2$ and $4$.  In a similar way one can show that $3$ is to big, but that's where the previously simple arithmetic gets messy.  Use the interval from $-1/6$ to $0$ for that.
Similarly with logarithms:
\begin{align}
\frac d {dx} \log_6 x & = \frac{\text{constant}} x, \\[12pt]
\text{ and } \quad \frac d {dx} \log_e x & = \frac{\text{constant}} x, \text{but this time, the constant is 1.}
\end{align}
What is natural about $e$ is the same thing that is natural about radians:
$$
\frac d {dx} (\text{sine of $x$ degrees}) = \Big((\text{cosine of $x$ degrees}) \times \text{constant} \Big)
$$
but only when radians are used is the "constant" equal to $1$. $($With degrees the constant is $\pi/180.)$
