Why don't fractals have more differentiable symmetries? Some sets tend not to "look" very homogeneous, such as self-similar fractals. I'd like to know why! And there's a particular class of statements that I'm hoping can be made...
Definition
Let $A$ be a subset of $\mathbb R^n$. Consider two points of $A$ to be equivalent if they have similar neighborhoods up to a differentiable map. Concretely, for $x,y\in A$, write $x\sim y$ if there exists a map $f:A\to A$ and an invertible matrix $T\in\mathbb R^{n\times n}$ such that $f(x)=y$ and
$$\lim_{\Delta x\to 0}\frac{f(x+\Delta x) - f(x)-T\Delta x}{|\Delta x|}=0.$$
I require $T$ to be invertible so that infinitesimal neighborhoods of $x$ and $y$ look affinely equivalent. (If we allowed $T$ to be singular, then $f$ could be a constant map, which wouldn't be interesting.) For that matter, let's go ahead and require that $f$ itself is a bijection between a neighborhood of $x$ and a neighborhood of $y$.
Example
Consider the standard middle-third Cantor set $C\subset[0,1]\subset\mathbb R^1$. This set is topologically homogeneous in the sense that for all $x,y\in C$, there is a homeomorphism $h:C\to C$ such that $h(x)=y$. In fact, if we consider $C$ to be a subset of $\mathbb R^2$, we can find an $h$ that extends to a homeomorphism of all of $\mathbb R^2$, and we can even find an isotopy from the identity on $\mathbb R^2$. So topological methods alone don't seem to be enough to distinguish between points in $C$.
Once we start considering differentiable maps, $C$ starts looking less homogeneous. In particular, $0\sim x$ if and only if $x$ has a finite ternary expansion. The equivalence class of $0$ is countable, whereas $C$ is uncountable, so there are relatively few points equivalent to $0$.
I suspect that every equivalence class in $C$ is countable, although I've only proven the restricted case of this latter statement when $f$ is required to be an isometry with respect to the natural ultrametric on $C$. I'm looking for a more general theory...
Questions
Under what conditions on $A$ are we assured that:


*

*All points in $A$ are equivalent? For example, it suffices that $A$ is an open set, or that $A$ is an embedded differentiable manifold.

*All equivalence classes in $A$ are at most countable? I can prove this for sufficiently nice maps $f$ where $A$ is a sufficiently nice embedding of the $p$-adic integers, which is the set that I'm most interested in, but again I'm looking for more general results. Does it suffice that $A$ doesn't contain any differentiable curves? Or that $A$ is nowhere a differentiable image of a product $\mathbb R^k\times B$?

*At least one equivalence class in $A$ is at most countable? For example, it suffices for $A$ to contain a corner point, as it can only be equivalent to other corner points, and there are at most countably many corners in any subset of $\mathbb R^n$. This generalizes the example of $0\in C$ given above. But some sets in $\mathbb R^n$ for $n\geq2$ don't have corners; what about them?


If necessary, you can assume that $A$ is closed, or even that $A$ is a Cantor set. You can also assume that the maps $f$ under consideration must be differentiable on a neighborhood of $x$, or even $C^1$, or that $f$ must be a bijection. A statement like "If $A$ is a Cantor set then every equivalence class is at most countable" would be a home run!
Is this an easy problem or a hard problem? I don't know what kind of theory the question belongs to, hence the reference-request tag. I've heard of the study of "analysis on Foo" where Foo = closed sets, metric spaces, or fractals, but these topics seem to focus on different sorts of questions. Of course, if this is a hard problem, I can also consult MathOverflow. But for all I know, the problem is an easy consequence of Theorem-I've-Never-Heard-Of.
(edited inline)
 A: I do not have a detailed answer, but there is a substantial literature on smooth rigidity of Cantor sets (and other dynamically defined fractals), starting with 
D. Sullivan, Differentiable structure on fractal-like sets determined by intrinsic scaling functions on dual Cantor sets.  Nonlinear evolution and chaotic phenomena (Noto, 1987), 101–110, NATO Adv. Sci. Inst. Ser. B Phys., 176, Plenum, New York, 1988.
and 
D. Sullivan, Differentiable structures on fractal-like sets, determined by intrinsic scaling functions on dual Cantor sets. The mathematical heritage of Hermann Weyl (Durham, NC, 1987), 15–23, Proc. Sympos. Pure Math., 48, Amer. Math. Soc., Providence, RI, 1988. 
See for instance: 
R. Bamón, C. Moreira, S. Plaza, J. Vera,
Differentiable structures of central Cantor sets.
Ergodic Theory Dynam. Systems 17 (1997), no. 5, 1027–1042. 
These papers mainly deal with smooth maps between different Cantor sets, but you should be able to use their results in the setting of a single Cantor set $C$ where you have the extra requirement that $f: C\to C$ sends $x$ to $y$, where $x, y$ are given points. 
If you go to www.ams.org/mathscinet and look for papers which refer to the two papers by Sullivan's listed above, you will find many more references.  
A: I've had better luck searching the literature for results related to the Hilbert-Smith conjecture. The frontier between possible and impossible seems to be somewhere between Lipschitz ambient homogeneity and $C^1$ ambient homogeneity.
On the positive side, from Repovs and Scepin (1997), "A proof of the Hilbert-Smith conjecture for actions by Lipschitz maps":

Malesic proved in 1994 that the standard Cantor set in R2 is Lipschitz ambient homogeneous. He also constructed Antoine’s necklace in R3 which is also Lipschitz ambiently homogeneous [18].

The reference [18] is to J. Malesic: Toroidal decompositions of the 3-dimensional sphere. Ph.D. Thesis, University of Ljubljana, 1995. I can't seem to find this reference, but I can guess with high confidence what they mean.
On the negative side, from Repovs, Skopenkov and Scepin (1996), "$C^1$-homogeneous compacta in $\mathbb R^n$ are $C^1$-submanifolds of $\mathbb R^n$":

We begin by recalling ... that a subset $K \subset \mathbb R^n$ is said to be $C^1$- homogeneous if for every pair of points $x, y \in K$ there exist neighborhoods $O_x, O_y \subset \mathbb R^n$ of $x$ and $y$, respectively, and a $C^1$-diffeomorphism
  $h: (O_x, O_x \cap K, x) \to (O_y, O_y \cap K, y)$, i.e. $h$ and $h^{-1}$ have continuous first derivatives...
Theorem 1.1. Let $K$ be a locally compact (possibly nonclosed) subset of $\mathbb R^n$. Then $K$ is $C^1$-homogeneous if and only if $K$ is a $C^1$-submanifold of $\mathbb R^n$.

I haven't digested the latter proof, but I think it leaves open the question of whether a Cantor set can be "merely differentiably homogeneous". I can now prove that this is impossible in $\mathbb R^n$ for $n=1$ and $n=2$ by elementary methods, although I don't have a proof for $n\geq 3$.
Further references or clarifications of these results would still be welcome!

Edit: I now have a proof for all $n$, to appear in Topology and its Applications as https://doi.org/10.1016/j.topol.2019.06.046 "Cantor sets are not tangent homogeneous"!
