Action of ambient isotopies on the homology of an embedded $n$-torus in $\mathbb{R}^{n+1}$ and $S^{n+1}$ that bring the $n$-torus back to itself Let $T^n$ be the $n$-torus embedded in $\mathbb{R}^{n+1}$. Consider the ambient isotopies that bring $T^n$ back to itself. These isotopies have an action on $H_1(T^n)$ described by some subgroup $G_n\leq \text{SL}_n(\mathbb{Z})$ unique up to conjugation due to dependence on choice of basis for $H_1(T^n)$. Similarly, let $G_n'$ be that for when we replace the ambient space by $S^{n+1}$ by one-point compactification of $\mathbb{R}^{n+1}$.
I would like to know what $G_n$ and $G_n’$ are for each $n$. It seems that $G_n$ and $G_n'$ may depend on choice of embedding of $T^n$, so I am primarily interested in when $T^n$ is the "unknotted" $n$-torus defined iteratively as the “surface” of revolution of $T^{n-1}$, where $T^1$ is the usual unit circle, but I would also like to know how $G_n$ and $G_n'$ change when we allow other embeddings of $T^n$. As far as for when $T^n$ is unknotted, I conjecture that $G_n$ is trivial for all $n$, and $G_n’$ is non-trivial for $n\geq 2$, but I have no idea how to prove this. All I have been able to show is that when we choose the canonical generators on the unknotted $T^2$, then $\left(\begin{smallmatrix}
  0 &1\\ 1 & 0\end{smallmatrix}\right)\in G_2'$, which can be seen by a rotation of the Clifford torus.
 A: Here's an answer just for $T^2 \subset \mathbb R^3$.
The complement $\mathbb R^3 - T^2$ has two components, a bounded component $B$ and an unbounded component $U$. The closure $\overline B = B \cup T^2$ is compact but $\overline U = U \cup T^2$ is noncompact. So any homeomorphism of $\mathbb R^3$ that takes $T^2$ to itself must take $\overline B$ to itself and $\overline U$ to itself.
Up to multiplication by $-1$ there is a unique generator $\beta \in H_1(T^2)$ which maps to the identity under the inclusion induced homomorphism $H_1(T^2) \mapsto H_1(\overline B)$. Also, up to multiplication by $-1$ there is a unique generator $\upsilon \in H_1(T^2)$ which maps to the identity under the inclusion induced homomorphism $H_1(T^2) \mapsto H_1(\overline U)$. Furthermore, $\beta,\upsilon$ form a $\mathbb Z$-basis of $H_1(T^2)$.
From this it follows that for any homeomorphism $f : T^2 \to T^2$ which is induced by an ambient isotopy of $\mathbb R^3$ that brings $T^2$ back to itself, we have $f_*(\beta) = \pm \beta$ and $f_*(\upsilon) = \pm \upsilon$.
So this restricts $G_2 \subset \text{Aut}(H_1(T^2)) \approx \text{GL}_2(\mathbb Z)$ to be a subgroup of the Klein 4-group acting on $H_1(T^2)$ by preserving $\{\pm\beta\}$ and preserving $\{\pm\upsilon\}$.
However, $f$ preserves the orientation of $\mathbb R^3$ and from this one can get rid of one degree of freedom, with the result that (up to change of sign on $\upsilon$) the only nontrivial possibility for $f$ is $\beta \mapsto -\beta$, $\upsilon \mapsto - \upsilon$.
This nontrivial possibility does indeed occur for the standard unknotted embedding $T^2 \subset \mathbb R^3$ which is rotationally symmetric around the $z$-axis: simply twirl the $x$-axis by $180^\circ$.
For general embeddings $T^2 \subset \mathbb R^3$ I don't have an answer in complete generality, although a key general feature is that $G_2$ is still restricted to be a subgroup of that same order 2 cyclic subgroup. If for example $T^2$ is the boundary of a tubular neighborhood of a knot $K \subset R^3$ then the "twirl" might or might not exist depending on $K$. I believe it does exist for torus knots, for example.

If you generalize to $T^2 \subset S^3$ then there are still two components $S^3 - T^2 = B \cup U$ but now both of $\overline B$, $\overline U$ are compact, and it is possible for them to be switched. This at first leads to a specific order $8$ subgroup of $\text{Aut}(H_1(T^2)) \approx \text{GL}_2(\mathbb Z)$ that must contain $G'_2$, although again orientation considerations whittle this down to a specific order $4$ subgroup. Again this entire order $4$ subgroup does occur for the standard unknotted embedding $T^2 \subset S^3$.
But for knotted embeddings $T^2 \subset S^3$ we are reduced to the $\mathbb R^3$ picture by the following considerations: for any embedding $T^2 \subset S^3$ at least one of $\overline B$, $\overline U$ is a solid torus, and for knotted embedding the other one is not a solid torus. Therefore, by removing one point from the interior of the one that is not a solid torus we are reduced to the case of a knotted embedding of $T^2$ in $\mathbb R^3$.
