$\sin(x)/x$ anti-derivative? There's this limit that I've seen its solution
$$\lim _{h\to 0}\left(\frac{1}{h}\int _{\frac{\pi }{4}}^{\frac{\pi }{4}+h}\left(\frac{\sin \left(x\right)}{x}\right)dx\:\right)$$
The solution starts with "Let $F(x)$ be the anti-derivative of $\frac{\sin(x)}{x}$ yada yada yada..."
I know that the anti-derivative of $\frac{\sin(x)}{x}$ isn't elementary, but is there a theorem that says that every elementary function has an anti-derivative? 
 A: The Fundamental Theorem of Calculus shows that every continuous function has an antiderivative. If $f$ is continuous on an interval containing $0$ and $$F(x)=\int_0^x f(t)\,dt$$then $F'(x)=f(x)$.
If you use that together with the definition of $F'(x)$ you see that this says $$\lim_{h\to0}\frac1h\int_x^{x+h}f(t)\,dt=f(x),$$which gives exactly the limit you ask about.
A: Yes, there is such a theorem. Every continuous function $f\colon U\to\mathbb{R}$, where $U\subseteq \mathbb{R}$ is an open set, has an anti-derivative, which you can even precisely write down as:
For any $a\in U$ the function $F_a\colon U\to\mathbb{R},\,x\mapsto \int^x_a f(t)\mathrm{d} t$ is an anti-derivative of $f$.
There are functions which do not have an anti-derivative which can be written as a composition of elementary functions, but this is only true as long as you don't consider the integral operator as an elementary function (which people usually don't). So yea, one can easily prove that for any continuous function the above function $F_a$ is differentiable and its derivative is $f$.
A: Yes.
This is a part of the Fundamental Theorem of Calculus (FTC). The FTC bidirectionally relates differentiation and integration. The first part tells you that antidifferentiation can be used to find an integral, in particular if you have a function $f$ with antiderivative $F := D^{-1}[f]$, then
$$\int_{a}^{b} f(x)\ dx = F(b) - F(a)$$
The second part tells you that the derivative of a definite integral is the given function, i.e. the definite integral defines an antiderivative:
$$\frac{d}{dx} \left[\int_{a}^{x} f(x')\ dx'\right] = f(x)$$.
Provided, of course, some suitable conditions on the behavior of $f$ are met, e.g. for the first part, continuity of $f$ is sufficient. This second part gives you what you want. $x \mapsto \frac{\sin(x)}{x}$ is smooth, even analytic, and so its antiderivative exists, and is equal to some integral of the above type.
FWIW, a specific antiderivative (namely the one which attains value 0 at $x = 0$) for this particular function has a name, it is called the sine integral, denoted $\mathrm{Si}(x)$ and taken as
$$\mathrm{Si}(x) := \int_{0}^{x} \frac{\sin(x')}{x'}\ dx'$$.
. The above theorem, suffices to make this definition correct. In fact the above theorem even guarantees this has an antiderivative as well, so it's not restricted to just "elementary" functions alone as the above function is not elementary, as you already mentioned. (In fact we can antidifferentiate arbitrarily many times.)
A: The antiderivative of a continuous function exists, which is exactly what the fundamental theorem of calculus says.
On the other hand, you don't need it for computing
$$
\lim_{h\to0}\frac{1}{h}\int_{a}^{a+h}f(x)\,dx=f(a)
$$
when $f$ is continuous in a neighborhood of $a$, for the simple reason that this is the fundamental theorem of calculus.
One of the most important properties of integrals of continuous functions is that, for $a<b$,
$$
(b-a)m_a^b(f)\le\int_a^b f(x)\,dx\le (b-a)M_a^b(f)
$$
where $m_a^b(f)$ and $M_a^b(f)$ are, respectively, the minimum and maximum value of $f$ over $[a,b]$. This can be rewritten as
$$
m_a^b(f)\le \frac{1}{b-a}\int_a^b f(t)\,dt\le M_a^b(f)
$$
and this also holds for $b<a$ (where $m_a^b(f)$ and $M_a^b(f)$ are the minimum and maximum of $f$ over $[a,b]$ or $[b,a]$, depending on $a<b$ or $a>b$). For $b=a+h$ (with $h\ne0$, it is easy to see that
$$
\lim_{h\to0}m_a^{a+h}(f)=f(a)=\lim_{h\to0}M_a^{a+h}(f)
$$
and this implies
$$
\lim_{h\to0}\frac{1}{h}\int_a^{a+h} f(t)\,dt=f(a)
$$
by sandwiching.
