Let $(M,\nabla^M),(N,\nabla^N)$ be two smooth manifolds with given (affine) connections on their (tangent bundles). We say a diffeomorphism ,$\phi:(M,\nabla^M)\rightarrow(N,\nabla^N)$ is an isomorphism if: $\nabla^N_X{Y}=\phi_* \left( \nabla^{M}_{\phi^{-1}_*(X)} {\phi^{-1}_*(Y)} \right) \forall X,Y \in \Gamma(TN)$,
where the pushforward $\phi_*(X)(q)=d\phi_{\phi^{-1}(q)}[X \left(\phi^{-1}(q)\right)]$ is the corresponding isomoprhism of Lie algebras.
Assume $(M,\nabla)$ is a smooth manifold with an affine connection (on its tangent bundle). Let $\phi \in \text{Diff(M)}$. If for every geodesic $\gamma$ (w.r.t $\nabla$) $\phi \circ \gamma$ is a geodesic, is it true that it must be an isomorphism of $(M,\nabla)$?
(Note that if we ask this question with two connections $\nabla_1,\nabla_2$ , and assume that $\phi$ maps geodesics $\nabla_1$ into geodesics of $\nabla_2$ , then clearly the answer is negative. For example, we can take two different connections with identical geodesics, and $\phi = Id$)