What I have in mind is along the lines of this:
Let $M$ a topological space, $V$ a normed vector space, and
$$ \boxminus \colon M\times M \to V, $$ $$ \boxplus \colon M\times V \to M. $$
Then $M$ is called a manifold1 with tangent space $V$, if $$ p \boxplus (q \boxminus p) = q, $$ $$ p \boxplus \vec{0} = p $$ and, for sufficiently small2 vectors $v,w$, $$ (p \boxplus v) \boxplus w \approx p \boxplus (v+w). $$
Can this set of axioms be made equivalent to the standard definition of manifolds (with an atlas of ball-homeomorphic neighbourhoods)?
Note that rather than requiring the manifold to be locally homeomorphic to a ball in a vector space, I require it to be globally isomorphic to some subset of a vector space, in a way that's locally a homeomorphism.
It sure seems to work for simple examples. Christian Blatter suggests $S^1$. Well, if we interpret that as $[0,1]$ with the ends glued together, then we can just define $\theta \boxminus \phi$ as $\theta - \phi$ folded back into the interval $[-\tfrac12, \tfrac12[ \subset V \equiv \mathbb{R}$, and $\theta \boxplus \delta$ as $\theta + \delta$ confined to $[0,1[$. The axioms hold; in fact $\boxplus$ is associative.
The same works for tori of arbitrary dimension. For $S^2$ it immediately gets a bit messier, but I'd figure something Euler-angle-ish should do the trick in much the same way as in $S^1$. Where I'm uncertain is with multi-hole handlebodies, or Klein bottles etc. – and even if all those work, with the general case.
1 Evidently, this definition implies at least that the manifold will be path-connected. Likely also other things, but which exactly?
2 A precise formulation of this: let $\varepsilon>0$, $p\in M$, $v,w \in V$. Then there exists $\eta>0$, such that $$\left\|\bigl((p \boxplus \eta v) \boxplus \eta w\bigr) \boxminus \bigl(p \boxplus (\eta v + \eta w)\bigr) \right\| < \eta\cdot \varepsilon.$$
