What is the limit of $\lim_{\Delta t \rightarrow 0}\sum_{k=0}^{T/\Delta t}(I + A\Delta t)^k \Delta t$ Let $A$ be an $n\times n$ matrix.
What is the limit (if any) of $\lim_{\Delta t \rightarrow 0}\sum_{k=0}^{T/\Delta t}(I + A\Delta t)^k \Delta t$?
My Attempt
What I tried so far is to say that $(I + A\Delta t)^k = \exp(Ak\Delta t) + O(\Delta t)$ in which case we can do 
$$\sum_{k=0}^{T/\Delta t}(I + A\Delta t)^k \Delta t = \sum_{k=0}^{T/\Delta t}[\exp(Ak\Delta t)\Delta t + O(\Delta t^2)] $$
$$= [\sum_{k=0}^{T/\Delta t}\exp(Ak\Delta t)\Delta t] + O(\Delta t)$$
$$= [\sum_{k=0}^{T/\Delta t}\exp(Ak\Delta t)\Delta t] + O(\Delta t)$$
$$\Delta t \rightarrow 0 \implies \int_0^T\exp(As)ds $$
My Attempt at a Lemma
But I can't seem to get a proof of $(I + A\Delta t)^k = \exp(Ak\Delta t) + O(\Delta t)$. I have the following but I couldn't complete it.
Using the matrix operator norm 
$$||\exp(Ak\Delta t) - (A\Delta t + I)^k|| = ||\sum_{i=0}^\infty (A k \Delta t)^i/i! - (A\Delta t + I)^k||$$
$$= ||\sum_{i=0}^\infty (A k \Delta t)^i/i! - (A\Delta t + I)^k||$$
$$= ||\sum_{i=0}^\infty (A k \Delta t)^i/i! - \sum_{i=0}^k {k \choose  i}(A \Delta t)^i||$$
For $k \ge 2$ the first and second terms of both sums are $I$ and $Ak\Delta t$ respectively and therefore they cancel.
$$\lt ||\sum_{i=2}^k (A k \Delta t)^i/i! -  {k \choose  i}(A \Delta t)^i|| + ||\sum_{i=k+1}^\infty (A k \Delta t)^i/i!|| $$
$$= ||\sum_{i=2}^k (A k \Delta t)^i/i! -  {k \choose  i}(A \Delta t)^i|| + ||\sum_{i=0}^\infty (A k \Delta t)^{i+k+1}/{(i+k+1)!}|| $$
$$= ||\sum_{i=2}^k (A k \Delta t)^i/i! -  {k \choose  i}(A \Delta t)^i|| + || (A k \Delta t)^{k+1}\sum_{i=0}^\infty (A k \Delta t)^{i}/{(i+k+1)!}|| $$
$$< ||\sum_{i=2}^k (A k \Delta t)^i/i! -  {k \choose  i}(A \Delta t)^i|| + ||\frac{ (A k \Delta t)^{k+1}}{(k+1)!}\sum_{i=0}^\infty (A k \Delta t)^{i}/{i!}|| $$
$$< ||\sum_{i=2}^k (A k \Delta t)^i/i! -  {k \choose  i}(A \Delta t)^i|| + ||\frac{ (A \Delta t)^{k+1} k^{k+1}}{(k+1)!}\exp(Ak\Delta t)|| $$
Because $k^{k+1}/(k+1)! \rightarrow 1$,  and $k\Delta t < T$, for sufficiently large $k$,
$$< ||\sum_{i=2}^k (A k \Delta t)^i/i! -  {k \choose  i}(A \Delta t)^i|| + O(\Delta t^{k+1})$$
And this is where I'm stuck.
 A: I finally massaged it out I think. In what follows I used $i$ as the index of summation instead of $k$. I redefined $k$ as $T/\Delta t$ (the max possible value of $i$)
Lemma A: $||\exp(A) - (A \Delta t + I)^{\Delta t^{-1}}|| \in O(\Delta t)$
$$=||\exp(A) - [\sum_{j=0}^{\Delta t^{-1}} {{\Delta t^{-1}} \choose j}(A \Delta t)^j]||$$
$$=||\exp(A) - [\sum_{j=0}^{\Delta t^{-1}} \frac{(\Delta t^{-1}) ... (\Delta t^{-1} - j + 1)}{j!}(A \Delta t)^j]||$$
$$=||\exp(A) - [\sum_{j=0}^{\Delta t^{-1}} \frac{(\Delta t\Delta t^{-1}) (\Delta t(\Delta t^{-1} - 1))... (\Delta t(\Delta t^{-1} - j + 1))}{j!}A^j]||$$
$$=||\exp(A) - [\sum_{j=0}^{\Delta t^{-1}} \frac{(1 - \Delta t)... (1 + \Delta t ( - j + 1))}{j!}A^j]||$$
$$=||\exp(A) - [\sum_{j=0}^{\Delta t^{-1}} \frac{(1 - \Delta t)... (1 + \Delta t ( - j + 1))}{j!}A^j]||$$
$$ \lt ||\sum_{j=2}^{\Delta t^{-1}} \frac{1-(1 - \Delta t)... (1 + \Delta t ( - j + 1))}{j!}A^j|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ \lt ||\sum_{j=2}^{\Delta t^{-1}} \frac{1-(1 - j\Delta t)^j}{j!}A^j|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
by concavity
$$ \lt ||\sum_{j=2}^{\Delta t^{-1}} \frac{j^2\Delta t}{j!}A^j|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ \lt \Delta t ||\sum_{j=2}^{\infty} \frac{2j(j-1)}{j!}A^j|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ = 2\Delta t ||\sum_{j=2}^{\infty} \frac{1}{(j-2)!}A^j|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ = 2\Delta t ||\sum_{j=0}^{\infty} \frac{1}{(j-2)!}A^{j+2}|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ \lt 2\Delta t \,||A^2||\,||\sum_{j=0}^{\infty} \frac{1}{j!}A^{j}|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ \lt 2\Delta t \,||A^2||\,||\exp(A)|| + ||\sum_{j=\Delta t^{-1} + 1}^\infty \frac{A^j}{j!} ||$$
$$ \lt 2\Delta t \,||A^2||\,||\exp(A)|| + \sum_{j=\Delta t^{-1} + 1}^\infty \frac{||A||^j}{j!} $$
and choosing $\Delta t^{-1} > 2||A||$
$$ \lt 2\Delta t \,||A^2||\,||\exp(A)|| + \sum_{j=\Delta t^{-1} + 1}^\infty (1/2)^j $$
$$ \lt 2\Delta t \,||A^2||\,||\exp(A)|| + \sum_{j=\Delta t^{-1} + 1}^\infty (1/2)^j $$
$$ \lt 2\Delta t \,||A^2||\,||\exp(A)|| + \int_{x=\Delta t^{-1}}^\infty 2^{-x}\,dx $$
$$ \lt 2\Delta t \,||A^2||\,||\exp(A)|| + (1/2)^{\Delta t^{-1}}/\ln2 $$
and choosing $\Delta t$ small enough that $\Delta t^{-1} \ln(1/2) + \ln \ln 2 < \ln(\Delta t)$
$$ \lt (2\Delta t \,||A^2||\,||\exp(A)|| + \Delta t) \in O(\Delta t)$$
Lemma A Q.E.D.
Lemma B:  $(A \Delta t + I)^i =  \exp(A i \Delta t) + O(\Delta t)$ for $0 \le i \le k$
Rewrite 
$$(A \Delta t + I)^i = ((A \Delta t + I)^{\Delta t^{-1}})^{(i/k)T} $$
Applying the Lemma A, 
$$((A \Delta t + I)^{\Delta t^{-1}})^{(i/k)T} = (\exp(A) + O(\Delta t))^{(i/k)T} $$
$$=\sum_{j=0}^{(i/k)T}{(i/k)T \choose j}\exp(A)^{(i/k)T - j} O(\Delta t)^{j}$$
$$=\exp(A (i/ k) T) + O(\Delta t)\sum_{j=1}^{(i/k)T}{(i/k)T \choose j}\exp(A)^{(i/k)T - j} O(\Delta t)^{j-1}$$
but for sufficiently small $\Delta t$,
$$||O(\Delta t)\sum_{j=1}^{(i/k)T}{(i/k)T \choose j}\exp(A)^{(i/k)T - j} O(\Delta t)^{j-1}||< || O(\Delta t) T{T\choose \frac{T}{2}}||$$
So we can fold the extra stuff into the $O(\Delta t)$ and finally get
$$(A \Delta t + I)^i=((A \Delta t + I)^{\Delta t^{-1}})^{(i/k)T}$$
$$ = \exp(A (i/ k) T) + O(\Delta t)$$
$$ = \exp(A i \Delta t) + O(\Delta t)$$  
Lemma B Q.E.D. 
