Seeking formal explanation of definite integral over infinitesimal interval Why does :
$$\frac{\displaystyle\int_t^{t+h}f(s) \, ds}{h} = f(t)\text{ as }h \rightarrow 0\text{ ?}$$
Intuitively this makes sense, because the value of the integral is infinitesimally close to $f(t)$, you have $h$ of them, and you divide the result by $h$.  However, I'd like a more formal explanation if possible.
 A: We will assume that $h$ approaches $0$ from the right. A similar argument deals with the general case. We will need to make some assumptions about $f$, since the result does not hold in general. We will show that if $f$ is continuous in some interval $[t,t+b]$, then
$$\lim_{h\to 0^+} \frac{1}{h}\int_t^{t+h} f(s)\,ds=f(t).$$
We need to assume (or prove) certain properties of the definite integral. Let $m_h$ be the minimum of $f(s)$ in the interval $[t,t+h]$, and let $M_h$ be the maximum of $f(s)$ in this interval. These both exist by continuity. 
From any of the standard definitions of the definite integral, we can show that
$$h\, m_h \le \int_t^{t+h} f(s)\,ds\le h\, M_h.$$
Divide through by $h$. We get
$$m_h \le \frac{1}{h} \int_t^{t+h} f(s)\,ds\le  M_h.$$
Now let $h\to 0^+$. Then $m_h\to f(t)$ and $M_h\to f(t)$, and now Squeezing gives us the desired result.  
Another way: Or else we can use more machinery, like the Fundamental Theorem of Calculus. Consider the function $g(h)$ given by
$$g(h)=\frac{\int_t^{t+h}f(s)\,ds}{h}. $$
Using L'Hospital's Rule and the Fundamental Theorem, we find that
$$\lim_{h\to 0}g(h)=\lim_{h\to 0}\frac{f(t+h)}{1}.$$
The limit on the right is $f(t)$. 
Remark: We stayed away from using the term "infinitesimal," since in the reals there is no positive element with is $\lt \frac{1}{n}$ for every positive integer $n$. One could reframe the first argument above using the language of non-standard analysis, but there seems to be no great advantage to doing so. 
A: If $f$ is continuous, then for every $\varepsilon>0$ there exists $\delta>0$ such that if $s$ differs from $t$ by less than $\delta$ then $f(s)$ differs from $f(x)$ by less than $\varepsilon$.  Consequently if $|h|<\delta$ then the integral $\displaystyle\int_t^{t+h} f(s)\,ds$ is an integral of a function whose values are between $f(t)\pm\varepsilon$, over an interval of length $h$.  Therefore the integral is between $h(f(t)\pm\varepsilon)$.  Divide by $h$ and the quotient is between $f(t)\pm\varepsilon$.
So no matter how small $\varepsilon$ is, we can find $\delta$ small enough that if $|h|<\delta$ then the expression whose limit is taken differs from $f(t)$ by less than $\varepsilon$.
