The deep reason why $\int \frac{1}{x}\operatorname{d}x$ is a transcendental function ($\log$) In general, the indefinite integral of $x^n$  has power $n+1$.  This is the standard power rule.  Why does it "break" for $n=-1$?  In other words, the derivative rule $$\frac{d}{dx} x^{n} = nx^{n-1}$$ fails to hold for $n=0$.
Is there some deep reason for this discontinuity?
 A: There are already great answers showing why $\int \frac{1}{x}=\ln(x) (+C)$.  But it's fun to ask why integrating other powers of $x$ does not produce a logarithmic answer.
We'll need integration by parts:
$$
\int u \frac{dv}{dx} dx = uv - \int v \frac{du}{dx} dx \ .
$$
Let's start by finding a result we'll need later, $\int x^m\ln(x)dx$, by making $u=\ln(x)$ and $\frac{dv}{dx}=x^m$ (i.e. $\frac{du}{dx}=\frac{1}{x}$ and $v=\frac{x^{m+1}}{m+1}$):
$$
\int x^m\ln(x)dx = \frac{x^{m+1}}{m+1}\ln(x) - \int \frac{x^{m+1}}{m+1} \frac{1}{x}dx
$$
$$
= \frac{x^{m+1}}{m+1}\ln(x) - \frac{x^{m+1}}{(m+1)^2} \ .
$$
Now we can work out $\int x^ndx$ by setting $u=x^{n+1}$ and $\frac{dv}{dx}=\frac{1}{x}$ (i.e. $\frac{du}{dx}=(n+1)x^n$ and $v=\ln(x)$).  We get
$$
\int x^ndx = \int x^{n+1}x^{-1}dx = x^{n+1}\ln(x) - (n+1)\int x^n\ln(x)dx
$$
If $n=-1$ then the entire second term vanishes because of the $(n+1)$ factor, so we are left with $x^{n+1}\ln(x) = x^{0}\ln(x) = \ln(x)$.  But in all other cases, we have to evaluate it by using our previous result:
$$
\int x^ndx = x^{n+1}\ln(x) - (n+1)\Bigg( \frac{x^{n+1}}{n+1}\ln(x) - \frac{x^{n+1}}{(n+1)^2} \Bigg)
$$
$$
= x^{n+1}\ln(x) - x^{n+1}\ln(x) + \frac{x^{n+1}}{(n+1)}
$$
$$
= \frac{x^{n+1}}{(n+1)} \ .
$$
We can interpret this as meaning that the $\ln(x)$s are always "somewhere" in the integration of powers of $x$, but in almost all circumstances they are cancelled out by another term.  For $x^{-1}$, uniquely, the cancelling term itself vanishes, and we are left with only the logarithm function.
Note that this argument was entirely circular, and thus doesn't "prove" anything.  And we should probably worry slightly about dismissing that second term before it is evaluated.  But I hope it was useful to see the problem from another direction.
Constructive criticism welcome.
A: The so-called "deep reason" is not deep at all.  The term $\displaystyle \log(x)$ is simply the constant term in the expansion of $\displaystyle \frac{x^{n+1}}{n+1}$ around $n=-1$.  To see this, we simply write
$$\begin{align}\frac{x^{n+1}}{n+1}&=\frac{e^{(n+1)\log(x)}}{n+1}\\\\&=\frac{1}{n+1}\sum_{k=0}^\infty \frac{(n+1)^k\,\log^k(x)}{k!}\\\\&=\sum_{k=0}^\infty\frac{(n+1)^{k-1}\log^k(x)}{k!}\\\\&=\frac{1}{n+1}+\log(x)+\frac12(n+1)\log^2(x)+O((n+1)^2)\end{align}$$
whence we see the leading terms in the asymptotic ($n\sim -1$) expansion of $\displaystyle \frac{x^{n+1}}{n+1}$.  Obviously, we see that $\lim_{n\to -1}\frac{x^{n+1}-1}{n+1}=\log(x)$.

More simply, let $f(x,n)$ be given by the integral
$$\begin{align}
f(x,n)&=\int_1^x t^n\,dt\\\\
&=\frac{x^{n+1}-1}{n+1}\tag 1
\end{align}$$
Note that $f(x,n)$ is continuos on $(0,\infty)\times \mathbb{R}$ (i.e., $n$ need not be restricted to the integers).
Then, note that the limit as $n\to -1$ of $f(x,n)=f(x,-1)$ is
$$\begin{align}
\int_1^x \frac1t\,dt&=f(x,-1)\\\\
&=\lim_{n\to -1}f(x,n)\\\\
&=\lim_{n\to -1} \frac{x^{n+1}-1}{n+1}\\\\
&=\lim_{n\to -1}\frac{e^{(n+1)\log(x)}-1}{n+1}\\\\
&=\log(x)
\end{align}$$
So, we can recover the expected result, $\int_1^x \frac{1}{t}\,dt=\log(x)$, by taking the limit in $(1)$ for $n\ne -1$.
A: The paradox disappears when you consider the antiderivatives having the common point $(1,0)$, i.e. the functions
$$\int_1^x t^{\alpha-1}dt=\frac{x^\alpha-1}\alpha,$$
which are plotted below for exponents $-\dfrac12,-\dfrac14,-\dfrac18,0,\dfrac18,\dfrac14,\dfrac12$.
The blue curve corresponds to $\alpha=0$ but is evaluated as the limit
$$\lim_{\alpha\to0}\frac{x^\alpha-1}\alpha=\ln(x).$$
It perfectly blends with the others.

A: $\int x^n = \frac {x^{n+1}}{n+1} +c$
clearly is undefined if $n = -1$ so the rule is breaking down.
but since $x^{-1}$ is bounded and continuous if $x>0$ the integral should exist.
Lets tackle this from the other direction.
What is $\frac {d}{dx} \log x$?
First, $e = \lim_\limits{n\to\infty}(1+\frac 1n)^n$
From the definition of derivative.
$\frac {d}{dx} \log x =$$ \lim_\limits {h\to 0} \frac {\log (x+h) - \log x}{h}\\
\lim_\limits {h\to 0} \frac {\log (\frac {x+h}{x})}{h} \\
\lim_\limits {h\to 0}\frac {\log (1+\frac hx)}h\\
\lim_\limits {h\to 0} \frac 1x \frac xh \log (1+\frac hx)\\
\frac 1x\lim_\limits {h\to 0}  \log (1+\frac hx)^\frac xh\\
\frac 1x \log (\lim_\limits {h\to 0} (1+\frac hx)^\frac xh)$
We can say $n = \frac xh$ and and as $h$ goes to $0, n$ goes to $\infty$
$\frac {d}{dx} \log x = \frac 1x\log e\\
\frac {d}{dx} \ln x = \frac 1x$
A: 
the indefinite integral of $x^n$  has power $n+1$

Note that this hold also for $cx^n$ for $c\neq0$.

the derivative rule $\frac{d}{dx} x^{n} = nx^{n-1}$ fails to hold for $n=0$.

As Newton already noted in a comment, it actually holds: $\frac{d}{dx} x^{0} = \frac{d}{dx} 1 = 0 = 0x^{-1}$  (if you consider $0^0=1$, the latter equality is only tru if $x\neq 0$).
So if you want an intuitive explanation, I would say the problem is that the pattern for the derivative rule introduces a constant factor of $0$ here that you cannot get rid of, i.e. "the indefinite integral of $cx^n$ has power $n+1$" is "true" for $n=-1$ only if $c=0$ to begin with, which covers only a degenerate case of $\int cx^{-1}$ and is not terribly helpful when evaluating $\int c'x^{-1}$ for $c'\neq 0$.
A: Short answer: In the formula
$$\frac{d}{d x} \left(\frac{x^a-1}{a}\right) = x^{a-1}$$
we take the limit $a\to 0$ and get
$$\frac{d}{dx} \left(\lim_{a->0} \frac{x^a-1}{a}\right) = x^{-1}$$
Long answer: We  cannot apply the formula
$$\int x^{a-1} d x = \frac{x^{a}}{a}+ \mathcal{C}$$ for $a=0$. However, we can find an anti derivative by integration. If we define $\log x$  for $x>0$ to be
$$\log x = \int_1^x \frac{dt}{t}$$ (the base point $0$ would not work a priori, so next best choice seems $1$)  then we have equality
$$ \int_1^x t^{-1}dt =\lim_{a->0} \int_1^x t^{a-1} dt  $$  that is
$$\log x = \lim_{a->0} \frac{x^a -1}{a}$$
So an antiderivative of $\frac{1}{x}$ is a limit of antiderivatives of $x^{a-1}$ as $a\to 0$. We only needed to fix the initial value at $x=1$ ( and not at $x=0$).
Now, the great thing about exponent $-1$ is the following formula
$$\int_a^b \frac{dt}{t} = \int_{Ma}^{Mb}\frac{dt}{t}$$
One can find a correspondence between Riemann sums of equal value (if a rectangle has base magnified by $M$ and height decreased by $M$ the area does not change). Therefore, we have
$$\log x + \log y = \int_1^x \frac{dt}{t}+ \int_1^y \frac{dt}{t}= \\
= \int_1^x \frac{dt}{t}+ \int_x^{xy} \frac{dt}{t}= \int_1^{xy} \frac{dt}{t} = \log(x y)$$
From this fundamental formula we get
$$\log(x^m) = m \log x$$ first for $m$ natural, then for $m$ integer, then rational, and finally for $m$ real (and $x>0$).
Define now by $e$ the unique number such that $\log e = 1$. Clearly $e>1$.
From the above we get
$$\lim_{a->0}\int_1^e t^{a-1} dt = \int_0 ^e t^{-1} dt$$ that is
$$\lim_{a->0} \frac{e^a-1}{a} = 1$$
In other words, $e$ is the unique positive number such that $e^a \approx 1+a$ for $a$ small.
Now, we get right away
$$\log(e^m) = m \log(e) = m$$ for every $m \in \mathbb{R}$.
A: I'll try to give a soft answer to what I see as the spirit of the question, which is not why you get exactly log, but why the behaviour is different when integrating $x^{k}$ for $k=-1$.
The way I see it is that the logarithm would actually be there for other powers of $x$ too, but it's "hidden" by the fact that in a geometric series one term "gobbles up" all the others together, except in the very special case when all terms are equal. Put in other words, the intuition is very close to the reason why $\sum_{i=a}^b c^i$ can be reasonably approximated by just the first term of the sum ($c^a$) if $c<1$, and just by the last ($c^b$) if $c>1$, regardless of how large $b-a$ is, i.e. of how many terms you have in the sum. But if $c=1$ no single term dominates all the others, and that's when you have to count them all, and you end up seeing the $b-a$ term "emerge" in $\sum_{i=a}^b 1^i=b-a+1$.
Informally, you can see how this applies to the case at hand writing $\int_{x_0}^{x_f} x^k dx$ as $\int_{x_0}^{2 x_0} x^k dx + \int_{2x_0}^{4x_0} x^k dx + ...$. Each of the $\approx \log_2 \frac{x_f}{x_0}$ terms has the same weight as the others if, and only if, $k$ has a very specific value (which?), in which case $\int_{x_0}^{x_f} x^k dx$ equals $\approx \log_2 \frac{x_f}{x_0}$ times $\int_{x_0}^{2 x_0} x^k dx$. If it's just an $\epsilon$ smaller, or larger, you get the sum within a constant factor of either $\int_{x_0}^{2 x_0} x^k dx$ or of $\int_{\frac{1}{2}x_f}^{x_f} x^k dx$, respectively, regardless of $\frac{x_f}{x_0}$. Note that instead of using $2$ as a base, we could have used $3$, or $e$, or $7.24$, or $\pi$, and the critical value of $k$ would have remained the same: it's the value that ensures that if you integrate $x^k$ over an interval $7.24$ longer, but with a starting point $7.24$ times larger, the integral does not change, $k=-1$.
This is actually a phenomenon that I've seen pop up really really often in math, physics, and computer science. You often have a sum of many terms, and a parameter that, for small values, makes the first term of the sum dominate all the others put together, and for large values, makes the last term dominate all the others put together, regardless of how many terms you have in the sum. But when the parameter equals exactly the critical value at which the transition between the two regimes occurs, all sort of strange quantities related to the number of terms in the sum appear in any approximation of it.
A: Take any real $r$, and any real parameter $x > 0$.
I believe the key to understanding the phenomenon in this question is to grasp:
$
\def\lfrac#1#2{{\large\frac{#1}{#2}}}
$


*

*Why $\lfrac{d(\ln(x))}{dx} = x^{-1}$, based on the definition of $\ln$ as the inverse of $\exp$ on $\mathbb{R}$.

*Why $\lfrac{d(x^r)}{dx} = r x^{r-1}$ and why it only allows us to obtain $\int x^r\ dx$ if $r \ne -1$.

*A combined explanation for $\int x^r\ dx$ that allows us to see where the cases split.
I shall demonstrate these three points in order from basic principles using basic properties of $\exp$ instead of 'ad-hoc' limit arguments. These properties are:


*

*$\exp' = \exp$.

*$\exp(x+y) = \exp(x) \exp(y)$ for any reals $x,y$.

*$x^r$ is defined as $\exp(r\ln(x))$. The reason for this is a topic for another post.

(1)
Let $y = \ln(x)$.
Then $x = \exp(y)$.   [By definition of $\ln$.]
Thus $\lfrac{dx}{dy} = \lfrac{d(\exp(y))}{dy} = \exp(y)$.   [Note that $\lfrac{d(\exp(y))}{dy}$ is defined since $x,y$ are bijectively related.]
And $\lfrac{dy}{dx} \lfrac{dx}{dy} = 1$.   [By the generalized chain rule.]
Thus $\lfrac{dy}{dx} = \exp(y)^{-1} = x^{-1}$.
(2)
$\lfrac{d(x^r)}{dx} = \lfrac{d(\exp(r\ln(x)))}{dx} = \exp(r\ln(x)) \lfrac{d(r\ln(x))}{dx} = x^r r x^{-1} = r x^{r-1}$.
(3)
To find $\int x^r\ dx$, we wish to find an anti-derivative for $x^r$ with respect to $x$.
From (1) and (2) we can already obtain the answer. But why? Note that in the above derivation it is the derivative of $\ln$ that is actually causing the power of $x$ to decrease in the derivative of $x^r$. Interesting, isn't it! Now one might say, wait a minute, doesn't the power decrease because of the binomial theorem in the standard proof where $(x+h)^r = x^r + r x^{r-1} h + \cdots$? But look again; we have proven (2) for arbitrary real $r$, so if you wish to use the binomial theorem instead you would have to have proven it for arbitrary real powers! That is non-trivial, and at its core requires proving (2) or solving some related differential equation or proving that term-wise differentiation for power series works!
So I would say that (1), rather than being a special case that fills the 'discontinuity' at $r = -1$ in the anti-derivative formula for $x^r$ with respect to $x$, is actually the underlying reason for the general case via the derivative of $x^r$ for arbitrary real $r$. Since $\ln$ is the sort of 'base case' here, it cannot be derived from the general case, and there is sort of a logically inevitable split.
A: A different approach is provided by considering the effects of substitution on the differential $x^n dx$. Turns out we can blame, or perhaps credit, our old friend zero!
Say we put $u=x^a$. Then $x^n=u^{n/a}$ and $dx=(1/a)u^{(1/a)-1)}du$. Thus:
$x^n dx=(n/a)u^{((n+1)/a-1)}du$
We can then trade any original exponent $n$ on $x$ for q different exponent $m$ on $u$ by selecting
$a=(n+1)/(m+1)$
or in product form
$a(m+1)=(n+1)$
Wait a minute, make that, we can almost interchange any $n$ for any $m$.  The proposed substitution method will fail for the specific case $n=-1$ unless we also accept $m=-1$. Likewise $m=-1$ cannot be accessed from any value of $n$ other than $-1$.  Note that this cutoff at $m=-1$ or $n=-1$ correlates with the multiplicative property of zero.
This means differentials having the form $x^n dx$ for different values of $n$ are interrelated by algebraic substitution and their integrals will be algebraic functions of each other -- except for $n=-1$ which is cut off from this interconvertibility.  So, the integral of specifically $x^{-1} dx$ will have a different, algebraically independent form versus all other power function integrals.  And it's all because of zero doing its own totalitarian thing with multiplication!
A: For me, the key observation has always been that $\int \frac{dx}{x}$ is "scale-free" in the following sense.  Suppose that you perform a scaling operation on $x$ by defining $u=ax$ for some constant $a$.  How does the integral change?  You get:
$$ \int \frac{dx}{x} = \int \frac{\frac{du}{a}}{\frac{u}{a}} = \int \frac{du}{u} $$
It doesn't change!  This would not be true for any other power of $x$, since the numerator and denominator would not cancel out perfectly.
This is fundamentally the same observation as the highest-voted answer by @Anonymous above, and there are hints of it in the other answers as well.  But I think that this way of explaining it is much more concise.
In physical terms, applying a scaling factor could represent measuring $x$ in feet rather than meters, for example.  It's unusual that the result does not depend at all on the units that you use, and this is ultimately the reason why the logarithm has all of its interesting properties, such as $\log ab = \log a + \log b$.
(For those who are interested, the term "scale-free" is a buzzword in statistical physics to describe some similar behavior in physical systems near a phase transition.  I have intentionally misused it here, because in physics you typically don't care about creating a constant factor when doing the scaling, so all power laws in physics are "scale-free," but here we are talking about strict equality.)
