For instance, I've just checked that that if you take the best linear approximation (in the $L^2$ sense) to a sufficiently nice function $f$ on the interval $[-\varepsilon, \varepsilon]$, and then let $\varepsilon \to 0$, you get $f(0) + x f'(0)$.
Surely we could make this stronger -- I imagine the analogous statements should hold for, say, the $L^1$ norm as well, or for most reasonable norms. Can we go farther, though?
Question: What is the strongest precise definition we can give the word "best" so that we have a statement of the form "the tangent line is the best linear approximation to a differentiable function"? (Feel free to replace "differentiable" with, say, $C^2$ or something if it makes for a more interesting answer.)
(Note: I'm aware of similar-sounding questions here, such as In what sense is the derivative the "best" linear approximation?, but the answers there don't answer my question.)