So I have heard about a lot of fancy numerical methods for solving ODEs. I know that there are methods that give asymptotically a low error like the Runge-Kutta methods. (Assuming sufficient smoothness.) These estimate the solution in a set of points $t_0$, $t_1$, etc. But what if I want to have a function that is close to the correct solution everywhere, not just in a discrete set of points?
I can extend the numerical solution to a piecewise linear function. This will be a continuous function and it will converge to the correct solution if the step-size goes to zero.
But the error estimate will be poor in most places unless I use a very small step-size, which rather defeats the purpose of using a high-order method. So how does one go about estimating the solution in practice between the $t_i$?