Spivak Chapter 14 Problem 9 Let $f(x)=\cos \frac{1}{x}$ for $x \ne 0$, and $f(x)=0$ for $x=0$. Is the function $F(x)=\int_0^x f$ differentiable at 0?
My progress:
$$\lim_{h\to0^+} \frac{F(h)}{h} = \lim_{h\to0^+} \frac{\int_0^h f}{h}=\lim_{h\to0^+}\frac{\cos(\frac{1}{h})}{1}$$, 
which does not exist.
We know that $$F'(0) = \lim_{h \to 0} \frac{F(h)}{h}$$. Therefore, the limit above should be equal to $0.$ 
However, the answer is that $F'(x)=0.$ 
What did I mess up?
 A: This is a tricky problem and Spivak has given a hint which you have perhaps overlooked. Consider the function $g(x) =x^2\sin(1/x),g(0)=0$ then it can be proved that $$g'(x) =2x\sin(1/x)-\cos (1/x), g'(0)=0$$ so that $$g'(x) =h(x) - f(x) $$ where $f$ is the function in your question and $$h(x) =2x\sin (1/x),h(0)=0$$ We have now via Fundamental Theorem of Calculus $$g(x) =g(x) - g(0)=\int_{0} ^{x} g'(t) \, dt=\int_{0}^{x}h(t)\,dt-\int_{0}^{x}f(t)\,dt$$ or $$F(x) =\int_{0}^{x}h(t)\,dt-g(x)$$ and using Fundamental Theorem of Calculus again we get $$F'(0)=h(0)-g'(0)=0$$ because $h$ is continuous at $0$.
A: Edits made: Your wanting to use the L’Hospital’s rule is fair, but your conclusion is not valid
You do find a zero numerator and denominator. So you are right to try progress by differentiating both. Edit: thanks to what is pointed out in the comments, the derivative is indeed $f(h)$. However, the limit does not exist. So you cannot conclude anything from L’H rule (see the section “requirement that the limit exist” here for an example). In brief, “L’H computations lead to a no-limit situation” does not imply the limit does not exist. 
As to why the derivative is zero, an outline: try writing the limit as $$\lim_{h \to 0} \frac{\int_0^h f}{h}=\lim_{M\to \infty} M\int_M^{\infty}\frac{\cos y}{y^2}dy $$
Now split the integral into intervals of length $2\pi$. In each interval, bound the positive part above (for example in $[2N\pi,(2N+1)\pi]$, we have $\frac{1}{y^2}\leq \frac{1}{(2N\pi)^2}$ and the negative part from below to show that M times the sum of integrals on these intervals is smaller than a constant times $1/M$. (There is a series sum involved). Reverse the bounds on the positive and negative parts to see that it is bounded below by a constant times $-1/M$. As $M \to \infty$, the limit is zero. 
There might be a more elegant way to show the effect of the cancellation, but that’s all I could work out for now. 
A: Where does your second equality come from? Why is the integral of $f(x)$ (you've lost a "dx") from $0$ to $h$ the same as $cos(1/h)$? The mean value theorem tells you it's the same as $\cos (1/c)$ for some $c$ between $0$ and $h$, but there's no guarantee that this $c$ is actually $h$. 
When you look at $\cos (1/c)$, you realize that this could be any number between $-1$ and $1$, so that this is not a profitable way to approach the problem. 
Also, "We know that $F'(x) = \lim \ldots$" is wrong; that should be "We know that $F'(0) = \ldots$". And the thing inside the limit should be 
$$\frac{F(h) - F(0)}{h}$$
although perhaps you simplified by evaluating $F(0)$. 
A: Naively one would say you can “apply L’Hopital” because you have 0/0, however L’Hopital is inconclusive if the ratio between the derivatives does not have a limit.
In order to show the derivative is zero, you need to check that, because of positive and negative cancellations you have the integral less than $c h$ for every $c$ provided $h$ is small enough.
You can try to show that using some change of variable.
