This is true for cubic polynomials $p$: If $p$ has local extrema, then the quadratic function $p'$ has roots and these roots are symmetric about $x = r$, where $r$ is the unique root of $p''$ (so that $(r, p(r))$ is an inflection point). Of course, the claim is vacuous for linear and quadratic polynomials.
On the other hand, the claim is in general false for polynomials of degree $> 3$: If a quartic polynomial has more than one extremum, it has three, and so $p'$ has three real roots. By translating, dilating, and reflecting (all of which preserve midpoints) we may as well assume that
$$p'(x) = x (x - r) (x - 1),$$
so that the extrema of $p$ occur at $0, r, 1$, where $0 < r < 1$.
At the midpoints $\frac{1}{2} r$, $\frac{1}{2} (r + 1)$ of consecutive roots, we compute that
$$p''\left(\tfrac{1}{2}r\right) = -\tfrac{1}{4} r^2 \qquad \textrm{and} \qquad p''\left(\tfrac{1}{2}(r + 1)\right) = -\tfrac{1}{4} (1 - r)^2,$$
which by hypothesis are never zero, so these are not inflection points. Thus, we can conclude for quartic polynomials that the claim never holds (at least in the nonvacuous case, that is, when the polynomial has multiple extrema).
The simple example $p(x) = x^5 - x$ shows that the claim can hold for specific quintic polynomials, but it fails in general in the degree $5$ case.