This is the weighted or extended version of the mean value theorem, $\int_a^bw(x)g(x)dx=g(c)\int_a^bw(x)dx$ if $w$ has a uniform sign. But you are right, one needs that $f$ is continuous for that.
Here we can show that the function $g$ in
is continuous on $[a,b]$ if $f$ is continuously differentiable there and we know from the interpolation error formula that its values are equal to $g(x)=\frac12f''(\xi_x)$ for some $\xi_x\in(a,b)$. There is no need for the relation $x\mapsto\xi_x$ to be continuous as well.
Proof of the continuity of $g$: Expanding the linear interpolation one finds an alternative way to write $g(x)$ as
so that for the limits of $g$ to exist in $a$ and $b$ we need that the difference quotients of $f$ converge there, that is, that $f$ is differentiable in these points.
Now inserted into the mean value theorem you get