A connection is a procedure for transporting geometric data across curves. An affine connection is a procedure for transporting tangent vectors specifically. More concretely, given any smooth curve $\gamma:[0,1]\to M$ and any vector $v\in T_{\gamma(0)}M$, an affine connection defines a unique vector $v'\in T_{\gamma(1)}$, which we can treat as in some sense "the same as" (or "parallel to") $v$. For this reason we call $v'$ the parallel transportation of $v$ along $\gamma$. We can also define a connection an affine connection in terms of a covariant derivative: the parallel transportation of $v$ is then the unique solution of $\nabla_{\dot\gamma(t)}v(t)=0$.
"Flatness" as it is used in differential geometry, is a property of connections. A connection is flat if parallel transportation along any closed curve is the identity. Equivalently, a connection is flat if parallel transportation is path independent. We can also define local flatness by applying the same criteria to paths contained in a neighborhood of each point. When we say a Riemannian manifold is flat, we mean that the Levy-Civita connection (which is uniqely determined by the mertic) is flat.
$\mathbb{R}^n$, as a Riemannian manifold with the standard metric, is flat. It is for this reason that we often don't think of tangent vectors in Euclidean space as as attached to points: we can freely identify them with tangent vectors at the origin without having to choose a specific path.
We say a Riemannian manifold $(M,g)$ is locally flat at $p$ if any of the following apply:
- The Levi-Civita connection on $M$ is locally flat at $p$.
- There is a neighborhood of $p$ which is isometric to a neighborhood in $\mathbb{R}^n$
- There is a set of local coordinates around $p$ in which the metric is just the. euclidean metric $g_{ij}=\delta_{ij}$ on the entire coordinate patch.
- There is a set of local coordinates in which the Christoffel symbols are identically zero on the entire coordinate patch.
- The Riemann curvature tensor is identically zero on a neighborhood of $p$.
On a manifold, $M$ there are many choices of metric, each of which comes with a Levi-Civita connection. The existence of a flat metric comes with several topological restrictions, and there are many manifolds which admit no flat metrics.
As an example, the 2-Sphere $S_2$ has no flat metrics: On a flat Riemannian manifold, we can extend every nonzero tangent vector to a smooth, nonvanishing vector field by transporting the tangent vector to every other point (the fact we can do this smoothly depends on path independence). On $S_2$, there are no such vector fields (by the hairy ball theorem). Therefore, $S_2$ does not admit a flat Riemannian structure. Additionally, since the $S_2$ is simply connected, all locally flat connections are globally flat, so it doesn't even admit a locally flat Riemannian structure.
There are also manifolds (such as the Möbius strip) which can be equipped with a locally flat Riemannian structure, but not a globally flat one, again due to topological obstrucions.
If we instead define general affine connections, we have more freedom to choose connection with torsion (the LC connection is by definition torsion free). In this case, there are two obstructions to local flatness: the Riemann tensor and the torsion tensor must be identically zero. This formulation of your question isn't that different, though, since every flat affine connection is a LC connection for some metric.
To address your question, then, the fact that a Riemannian manifold is (locally) flat can tell you quite a bit about its topology, but it doesn't uniquely determine it. Conversely, we can only construct flat affine connections on manifolds with specific topologies.
I'm not familiar enough with statistical manifolds to comment on your last point.
