$M$ complete, simply connected manifold with zero curvature $\Rightarrow M$ is globally isometric to $\mathbb{R}^n$ 
Let $(M^n,g)$ be a complete, simply connected Riemannian manifold. If the sectional curvature is zero everywhere, then $M$ is globally isometric to $\mathbb{R}^n$ with the standard metric.

This certainly has to do with Hadamard's theorem, which in this case guarantees that for some $p\in M$ the exponential map $\text{exp}_p:T_pM\to M$ is a diffeomorphism.
I know that using the chart $\text{exp}_p^{-1}:M\to T_pM$ we have that:
$$g_{ij}(p)=\delta_{ij}$$
$$\frac{\partial}{\partial x_\ell}g_{ij}(p)=0$$
This means that $g$ is identical to the standard metric in $\mathbb{R}^n$ in $2^\text{nd}$ order approximation. 
Is it possible to argue that the higher order terms must be all zero?
If not, what other argument could I use?
 A: I've just found out this is basically exercise $5.1$ from do Carmo's Riemannian Geometry.
The only difference here is the additional information that $\exp_p$ is a diffeomorphism defined on all $T_pM$. To prove $\exp_p$ is an isometry, we need $2$ results:

$1)$ If $M$ has constant curvature $K$, the Jacobi equation becomes $J''+KJ=0$ (in this case, $J''=0$, so $J(t)=t\cdot w(t)$, where $w(t)$ is a parallel field along the geodesic);
$2)$ If $\gamma(t):=\exp_p(tv)$ and $J$ is a Jacobi field along $\gamma$ with $J(0)=0$, then $J(t)=(d\exp_p)_{tv}(tw)$ for some $w\in T_v(T_pM)$.

Now take arbitrary $v\in T_pM$ and $w_1,w_2\in T_v(T_pM)\equiv T_pM$.
Let $w_i(t)$ be the parallel transport of $w_i$ along $\gamma:t\mapsto \exp_p(tv)$ and $J_i(t):=(d\exp_p)_{tv}(tw_i)$. Clearly $J_i(0)=0$ and $J_i'(0)=w_i$, so by uniqueness $J_i(t)=t\cdot w_i(t)$. Therefore:
\begin{align*}
\langle (d\exp_p)_v(w_1),(d\exp_p)_v(w_2)\rangle &=\langle J_1(1),J_2(1)\rangle_{\gamma(1)}\\
&=\langle w_1(1),w_2(1)\rangle_{\gamma(1)}\\
&=\langle w_1,w_2\rangle_p
\end{align*}
(in the last equation we use the invariance of $\langle\cdot,\cdot\rangle$ under parallel transport) $_\blacksquare$
