Characterization of $C^{k,\alpha}$ (functions with Hölder continuous derivatives) through Taylor estimates For $k \in \Bbb{N} = \{1,2,3,\dots\}$ and $\alpha \in (0,1)$, let us define
$$
C^{k,\alpha} := \{ f : \Bbb{R} \to \Bbb{R} \,:\, f \in C^k \text{ with } f, f', \dots, f^{(k)} \text{ bounded and } [f^{(k)}]_{C^\alpha} < \infty \},
$$
where
$$
[g]_{C^\alpha} := \sup_{x,y \in \Bbb{R}, x \neq y} \frac{|g(x)-g(y)|}{|x-y|^\alpha} \, .
$$
It is not hard to see by using Taylor's formula for $f \in C^{k,\alpha}$ that for each $x \in \Bbb{R}$, there is a polynomial $P_x$ of degree $k$ (namely, the Taylor polynomial of $f$) satisfying
$$
|f(x+h) - P_x (h)| \leq C \cdot |h|^{k+\alpha} \qquad \forall h \in \Bbb{R} \text{ with } |h| \leq 1, \tag{$\circledast$}
$$
with a constant $C$ independent of $x,h$ (but depending on $f$).

It is claimed in Exercise 13.31 of certain notes by Martin Hairer that also the converse holds, i.e., if $f : \Bbb{R} \to \Bbb{R}$ and if there is a constant $C>0$, such that for each $x$, there is a polynomial $P_x$ of degree at most $k$ such that $(\circledast)$ holds, then $f \in C^{k,\alpha}$.

I would like to know a solution to this exercise, and even the case $k=1$ would be interesting to me.
To simplify the question a bit, let us assume that we already know $f$ to be $k$-times differentiable, with $f,f', \dots, f^{(k)}$ being bounded (I think this boundedness does not actually follow "automatically"), so we only need to establish $\alpha$-Hölder continuity of $f^{(k)}$.
After the fold, I discuss some of my ideas towards this problem. It might be better, however, to at first not read these ideas, since given that this is an exercise, they seem overly complicated (they refer to Campanato spaces, which are probably not known to everyone doing the exercise). Thus it might be better to start afresh instead of getting spoiled by my stupid thoughts :)

My thoughts only apply to the case $k=1$. Here, one can more or less easily show that necessarily $P_x (h) = f(x) + f'(x) h$.
Then we get
\begin{align*}
& |f(x+h) - f(x-h) - 2h \cdot f'(x)| \\
& = |f(x+h) - f(x) - f'(x) h - (f(x-h) - f(x) - f'(x) \cdot (-h))| \\
& \leq 2C |h|^{1+\alpha}
\end{align*}
for $|h| \leq 1$. But using the fundamental theorem of calculus to write $f(x+h) - f(x-h) = \int_{x-h}^{x+h} f'(s) ds$, and writing $g = f'$, we thus get
$$
\left|
- \int_{x-h}^{x+h} g(s) - g(x) ds
\right|
\leq 2C \cdot |h|^{1+\alpha}. \tag{$\dagger$}
$$
Now, it is known (see Lemma 0.0.15 in Some notes on Campanato spaces) that the norm
$$
\| g \|_\ast = \|g\|_{L^1} + \sup_{x \in \Bbb{R}, 0 < h \leq C} \,\, \inf_{c \in \Bbb{R}} \,\, h^{-(1 + \alpha)} \int_{x-h}^{x+h} |g(t) - c| \, dt
$$
is an equivalent norm for the Campanato space $L^{1,1+\alpha}_C (\Bbb{R})$. Strictly speaking, I am cheating a bit here, since $\Bbb{R}$ is not a bounded domain.
Thus, if in $(\dagger)$ above, we would have the absolute value inside the integral instead of outside, we would get a Campanato condition on the function $g = f'$. But it is known (see Wikipedia, or Theorem 0.0.22 of the notes I linked above) that $L^{1, 1+\alpha}_C = C^{0,\alpha}$ is the space of $\alpha$-Hölder continuous functions.
Thus, my hope is (I am currently investigating this) that one can adapt the arguments from the lecture notes above to also work in the case where the absolute value is outside of the integral.

As I said above, I don't think that this is the right solution, since it seems overly complicated :) I would be happy about any thoughts!

 A: We have
$$
|f(x+h)-P_{x}(h)|\leq c|h|^{k+\alpha}%
$$
for all $x\in\mathbb{R}$ and all $h$ with $|h|\leq1$. Write
$$
P_{x}(h)=\sum_{n=0}^{k}\frac{1}{n!}a_{n}(x)h^{n}.
$$
Taking $h=0$ you get $f(x)=a_{0}(x)$ so
$$
\left\vert f(x+h)-f(x)-\sum_{n=1}^{k}\frac{1}{n!}a_{n}(x)h^{n}\right\vert \leq
c|h|^{k+\alpha}.
$$
Next, by the triangle inequality
\begin{align*}
\left\vert f(x+h)-f(x)-a_{1}(x)h\right\vert  & \leq\left\vert f(x+h)-f(x)-\sum
_{n=1}^{k}\frac{1}{n!}a_{n}(x)h^{n}\right\vert +\sum_{n=2}^{k}|a_{n}%
(x)||h|^{n}\\
& \leq c|h|^{k+\alpha}+\sum_{n=2}^{k}\frac{1}{n!}|a_{n}(x)||h|^{n}%
\end{align*}
and so%
$$
\left\vert \frac{f(x+h)-f(x)-a_{1}(x)h}{h}\right\vert \leq c|h|^{k-1+\alpha
}+\sum_{n=2}^{k}\frac{1}{n!}|a_{n}(x)||h|^{n-1}\rightarrow0
$$
as $h\rightarrow0$, which shows that $f$ is differentiable with $f^{\prime
}(x)=a_{1}(x)$. 
%%%%%This part feels wrong but I cannot find the mistake%%%%%%%%%%%%
Hence, for $k=1$,
$$
\left\vert f(x+h)-f(x)-f^{\prime}(x)h\right\vert \leq c|h|^{1+\alpha}.
$$
In turn, replacing $x$ with $x-h$ we get
$$
\left\vert f(x)-f(x-h)-f^{\prime}(x-h)h\right\vert \leq c|h|^{1+\alpha}%
$$
while replacing $h$ with $-h$ gives
$$
\left\vert f(x-h)-f(x)+f^{\prime}(x)h\right\vert \leq c|h|^{1+\alpha}.
$$
Now
\begin{align*}
|f^{\prime}(x)-f^{\prime}(x-h)|  & \leq\left\vert \frac{f(x-h)-f(x)}
{h}+f^{\prime}(x)\right\vert +\left\vert -\frac{f(x-h)-f(x)}{h}-f^{\prime
}(x-h)\right\vert \\
& =\left\vert \frac{f(x-h)-f(x)}{h}+f^{\prime}(x)\right\vert +\left\vert
\frac{f(x)-f(x-h)}{h}-f^{\prime}(x-h)\right\vert \\
& \leq2c|h|^{\alpha}.
\end{align*}
This proves that $f^{\prime}$ is Holder's continuous. 
%%%%%%%%%%%%%%%End funny part %%%%%%%%%%%%
For $k\geq2$ we first prove that all the functions $a_{n}$ are bounded in
$[a,b]$. We will use the following fact about polynomials. Given
$k\in\mathbb{N}$, let $V$ be the vector space of all polynomials
$P:[-1,1]\rightarrow\mathbb{R}$ of degree less than or equal to $k$. Given
$P\in V$, let $\Vert P\Vert:=\max\{|a_{0}|,\ldots,|a_{k}|\}$, where
$P(t)=a_{0}+\cdots+a_{k}t^{k}$, $t\in\lbrack-1,1]$. Then $\Vert\cdot\Vert$ is
a norm in $V$. 
Since the vector space $V$ has finite dimension $k$, all norms are equivalent.
In particular,%
$$
c_{1}\Vert P\Vert_{L^{\infty}([-1,1])}\leq\Vert P\Vert\leq c_{2}\Vert
P\Vert_{L^{\infty}([-1,1])}%
$$
for all $P\in V$ and for some constants $c_{1}>0$ and $c_{2}>0$. 
Now let $P\in V$ be such that $\Vert P\Vert_{L^{\infty}([-1,1])}\leq L$. Then
by the previous inequality,%
$$
\Vert P\Vert:=\max\{|a_{0}|,\ldots,|a_{k}|\}\leq c_{2}\Vert P\Vert_{L^{\infty
}([-1,1])}\leq c_{2}L,
$$
which implies that $|a_{n}|\leq c_{2}L$ for all $n=1, \ldots, k$. 
Since $f$ is continuous, there exists
$$
M=\max_{[a-1,b+1]}|f(x)|<\infty
$$
and so for $-1\leq h\leq1$ we have
\begin{align*}
\left\vert \sum_{n=1}^{k}\frac{1}{n!}a_{n}(x)h^{n}\right\vert  &
\leq\left\vert f(x+h)-f(x)-\sum_{n=1}^{k}\frac{1}{n!}a_{n}(x)h^{n}\right\vert
+\left\vert f(x+h)-f(x)\right\vert \\
& \leq c|h|^{k+\alpha}+2M\leq c+2M.
\end{align*}
Hence,
$$
\left\vert \sum_{n=1}^{k}\frac{1}{n!}a_{n}(x)h^{n}\right\vert \leq c+2M
$$
for all $h\in\lbrack-1,1]$. It follows from the property about polynomials
above that there exists a constant $C=C(a,b,M,c)>0$ such that%
$$
|a_{n}(x)|\leq C
$$
for all $x\in\lbrack a,b]$ and all $n=1,\ldots,k$.
We can now apply Theorem 3 in Oliver's paper to conclude that $a_{n}%
(x)=f^{(n)}(x)$. 
