Taylor's theorem with remainder of fractional order? Let $k\geq 1$. Consider Taylor's theorem.
We know the Peano form and the mean-value form of the remainder term:
Peano form of the remainder
Let $f\colon (-\varepsilon,\varepsilon)\to\mathbb R$ be $k$ times differentiable. Then it holds
\[ f(h) = \sum_{m=0}^k \frac{f^{(m)}(0)h^m}{m!} + o(h^k). \]
Mean-value form of the remainder
Let $f\colon (-\varepsilon,\varepsilon)\to\mathbb R$ be $k+1$ times differentiable. Then it holds
\[ f(h) = \sum_{m=0}^k \frac{f^{(m)}(0)h^m}{m!} + \mathcal O(h^{k+1}). \]
Another form of remainder?
The question is, is there a theorem whose proposition is
\[ f(h) = \sum_{m=0}^k \frac{f^{(m)}(0)h^m}{m!} + \mathcal O(h^{k+\delta}) \]
with some $\delta\in(1,0)$. If so, what are sufficient assumptions?
Trivially, my proposition holds if $f$ is $k+1$ times differentiable, but I want a slightly weaker assumption.
 A: This partial answer might shed some light here.
Without loss of generality, we can examine the case where $k = 1$ and the interval is $[0,\epsilon).$
Can we find a continuous and differentiable function $f:[0,\epsilon) \to \mathbb{R}$ with the following behavior? 
The Taylor expansion is $f(h) = f(0) + f'(0)h + R(h)$, and for all $0 < \delta < 1$ we have
$$R(h) = \mathcal O(h^{1 + \delta}),$$
but
$$R(h) \neq \mathcal O(h^{2}).$$
An example is
$$f(x) = \begin{cases}x^2 \log x, &\mbox{if } x > 0 \\ 0, &\mbox{if } x = 0\end{cases}$$
with
$$f'(x) = \begin{cases}2x \log x + x, &\mbox{if } x > 0 \\ 0, &\mbox{if } x = 0\end{cases}$$
In this case, the function is itself the remainder of a first-order Taylor expansion, 
$$f(h) = f(0) + f'(0)h + R(h) = R(h),$$
and the desired behavior of the remainder follows 
$$\lim_{h \to 0} \left|\frac{h^2 \log h}{h^{1+ \delta}}\right| =\lim_{h \to 0} h^{1-\delta} \log h = 0, \\ \lim_{h \to 0}  \left|\frac{h^2 \log h}{h^2}\right| = \lim_{h \to 0}  |\log h| = \infty.$$
Under what general conditions do we get this asymptotic behavior?
In the given example, $f'$ is continuous on $[0,\epsilon)$ and $f''$ exists on $(0,\epsilon):$
$$f''(x) = \begin{cases}2 \log x + 1, &\mbox{if } x > 0 \\ -\infty, &\mbox{if } x = 0\end{cases}.$$
Clearly $f''$ fails to exist at and is unbounded in a neighborhood of $0$. However, $f''$ is integrable.  In this case, we can examine the integral form of the remainder
$$R(h) = \int_0^h (h-t) f''(t) \, dt.$$
If $f''$ is bounded and integrable then it is easy to show $R(h) = \mathcal O(h^{2}).$
I would conjecture that $f''$ unbounded and integrable  is at least necessary for this asymptotic behavior.
A: I have found a proof (thanks to the light that @RRL shed) that I will present here:
Slight generalization of Taylor's theorem
Let $\varepsilon>0$, $I=(-\varepsilon,\varepsilon)$,
$1<k\in\mathbb N$, $p>1$ and $f\in C^k(I)$, as well as $f^{(k)}\in W^{1,p}(I)$.
This means
$$ f\colon (-\varepsilon,\varepsilon) \to \mathbb R $$
is $k$-times continuously differentiable and the derivative $f^{(k)}$ is
weak differentiable. Thus, $f$ and its derivatives up to degree
$k$ are Lebesgue integrable and the weak derivative
of degree $k+1$ lies in $L^p(I)$.
Then it holds for $h\in I$
$$ f(h) = \sum_{m=0}^k \frac{f^{(m)}(0) h^m}{m!} + R(h), $$
such that for $h\to 0$ it holds
$$ R(h) \in\mathcal O\left(|h|^{k + 1 - \frac1p}\right). $$
Proof
First, two small Lemmata:
Lemma on weak integration by parts
Let $x\in C^\omega(I)$ analytic and $y\in W^{1,p}(I)$ weakly differentiable
and $a,b\in I$. Then it holds
\begin{align*}
   \int_a^b x'(t) y(t)\, dt = x(t)y(t)\big|_a^b - \int_a^b x(t) y'(t)\, dt.
\end{align*}
The integrals are meant as Lebesgue-Integrals.
Proof, see here.
Lemma on iterated integration by parts
Let $f\in C^k(I)$, then it holds
\begin{align*}
   f(h) = \sum_{m=0}^{k-1} \frac{h^m}{m!}f^{(m)}(0) + \frac1{(k-1)!}\int_0^h(h-t)^{k-1} f^{(k)}(t)\,dt
\end{align*}

Proof

By induction. Base clause ($\ell=1$) is the fundamental theorem of calculus:
\begin{align*}
   f(h) = f(0) + \int_0^h f'(t)\, dt = \sum_{m=0}^{0} \frac{h^m}{m!}f^{(m)}(0) + \frac1{0!}\int_0^h(h-t)^{0} f^{(1)}(t)\,dt.
\end{align*}
Suppose the Lemma is true for $\ell\geq 1$. Then it holds using integration
by parts:
\begin{align*}
   f(h) - \sum_{m=0}^{\ell-1} \frac{h^m}{m!}f^{(m)}(0) &=  \frac1{(\ell-1)!}\int_0^h(h-t)^{\ell-1} f^{(\ell)}(t)\,dt \\
        &= \frac1{\ell!}(-1)(h-t)^{\ell} f^{(\ell)}(t)\bigg|_0^h - \frac1{\ell!}\int_0^h (-1)(h-t)^\ell f^{(\ell+1)}(t)\,dt \\
        &= \frac1{\ell!}h^{\ell} f^{(\ell)}(0) + \frac1{\ell!}\int_0^h (h-t)^\ell f^{(\ell+1)}(t)\,dt.
\end{align*}
Proof of the theorem
First, use the Lemma on iterated integration by parts. This Lemma is known
from the proof of the integral remainder theorem.
Now we apply the induction step of the Lemma again, keeping in mind the
Lemma on weak integration by parts:
\begin{align*}
   f(h) &= \sum_{m=0}^{k} \frac{f^{(m)}(0) h^m}{m!} + \underbrace{\frac1{k!}\int_0^h (h-t)^{k} f^{(k+1)}(t)\, dt}_{=R(h)}.
\end{align*}
Consider for $h>0$ (analog $h<0$) the remainder $R$ with $\frac1p + \frac1q = 1$
and apply Hölders Inequality:
\begin{align*}
   |R(h)| &\leq \frac1{k!} \int_0^h \bigl| (h-t)^k f^{(k+1)}(t)\bigr|\, dt \\
          &\leq \frac1{k!} \left(\int_0^h |h-t|^{kq}\,dt\right)^{\frac1q} \left(\int_0^h |f^{(k+1)}(t)|^p\right)^{\frac1p}\\
          &\leq \frac1{k!} \left(\frac{|h|^{1+kq}}{1+kq}\right)^{\frac1q} \|f^{(k+1)}\|_{L^p(I)} \\
          &= |h|^{k+1-\frac1p} \cdot \frac{\|f^{(k+1)}\|_{L^p(I)}}{k!(1+kq)^{\frac1q}} = \mathcal O\left(|h|^{k+1-\frac1p}\right)
\end{align*}
