Existence of 8-fold symmetric domains in higher dimensions Let a nonempty set $A \subset \mathbb{R}^N$ (with $N \geq 3$) be bounded, open, and connected. (Preferably, the boundary of $A$ has some reasonable smoothness, say, Lipschitz.)
Suppose that $A$ is neither a ball nor a spherical shell.

Can such $A$ be 8-fold rotationally symmetric with respect to any pair of coordinate vectors $x_i$ and $x_j$?

Notice that in the case $N=2$ this is evidently true.
Moreover, the answer would be also affirmative for any dimension if the assumption that $A$ is open and connected is dropped, see an explanation here.
On the other hand, under the imposed assumptions, it seems to me that in the case $N=3$ the existence of such $A$ is not possible. See also a related question.
 A: In fact, if $A\subset {\mathbb R}^n$ is an open nonempty connected subset which has an 8-fold rotational symmetry in every coordinate plane, then $A$ is either a ball (possibly of infinite radius) or a spherical shell (again, of possibly infinite radius).
Let me first make your question a bit more precise. Consider ${\mathbb R}^n$ with the standard inner product $\langle .,.\rangle$ and the corresponding special orthogonal group  $G=SO(n)$ consisting of linear transformations with unit determinant preserving $\langle .,.\rangle$. Given a linear subspace $V\subset {\mathbb R}^n$ let $G_V$ denote the subgroup of $G$ consisting of orthogonal transformations preserving $V$ and fixing the orthogonal complement of $V$ element-wise. I will use the notation $V_{ij}$, $V_{ijk}$ to denote the subspace of $V$ spanned by the coordinate vectors $e_i, e_j$ (for $V_{ij}$) and $e_i, e_j, e_k$ (for $V_{ijk}$): I do not like using the notation $x_i$ for coordinate vectors.
With these notation, your question becomes:

Suppose that $A\subset {\mathbb R}^n$, $n\ge 3$, is an open connected nonempty subset such that for each pair of distinct indices $i, j\in \{1,...,n\}$, there exists an element of order $8$ in $G_{ij}:=G(V_{ij})$ preserving $A$. Is it true that $A$ is either a ball or a spherical shell? I.e., is it true that either $A$ has the form
$$
A=\{x: |x|< R\}
$$
for some $R$, $0< R \le \infty$ or the form
$$
A=\{x: r< |x|< R\}
$$
for some $r, R$ satisfying $0\le r< R\le \infty$?

This question has positive answer. However, a proof requires some degree of comfort with the theory of Lie groups.
I will prove, in fact, more. Let $H=G_A$ denote the subgroup of $G$ consisting of elements preserving $A$. This subgroup is necessarily closed, since it is the stabilizer of the closed subset ${\mathbb R}^n \setminus A$ in $G$. Hence, $H$ is a compact Lie subgroup of $G$ (this is a special case of a
theorem due to Cartan: closed subgroups of Lie groups are Lie subgroups).
Theorem. Suppose that $H\le G=SO(n), n\ge 3$, is a closed subgroup containing an element of order $8$ in each subgroup $G_{ij}$. Then $H=G$.
Proof. 1. Consider first the case $n=3$. Then, according to the classification of finite subgroups $\Phi$ of $SO(3)$, if $\Phi$ contains an element of order $8$, then $\Phi$ is either cyclic or dihedral and preserves a plane in ${\mathbb R}^3$. (The classification was discussed many times at MSE, see e.g. here.) Since we are assuming that $H\le SO(3)$ contains an order 8 rotation in each $G_{ij}< SO(3)$, such $H$ cannot preserve any plane: If $P$ were an invariant plane, one of the coordinate subspaces $V_{ij}$ would intersect $P$ along a line $L$. An order 8 rotation preserving $V_{ij}$ would send $L$ to another line $L'\subset V_{ij}$ different from $L$. Hence, $L'$ cannot be in $P$, which means that $P$ cannot be preserved by $H$.
Hence, such $H$ cannot be finite. Therefore, $H$ has positive dimension. Let $H_0\le H$ be the connected component of the identity in $H$, it is necessarily a normal subgroup of $H$. The subgroup $H_0$ has to have positive dimension (since its dimension equals that of $H$). It is an easy exercise to prove that the Lie algebra $o(3)$ contains no 2-dimensional subalgebras. Hence, $H_0$ has dimension 1 or 3. If $H_0$ is 3-dimensional, its Lie algebra is $o(3)$ (the Lie algebra of $SO(3)$). Since the exponential map for compact connected Lie groups is surjective, it follows that in this case $H_0=G$ and, thus, $H=G$. If $H_0$ is 1-dimensional, it is a subgroup of rotations  $G_V$ for some plane $V\subset {\mathbb R}^3$. Since $H_0$ is normal in $H$, the invariant plane $V$ has to be invariant under $H$ as well, which contradicts our assumption about order 8 elements (see above). Thus, if $n=3$, then $H=G=SO(3)$ indeed.


*The general case. Consider a 3-dimensional subspace  $V_{ijk}\subset  {\mathbb R}^n$. Since $H\le G$ contains order 8 rotations in the subgroups $G_{ij}, G_{jk}, G_{ki}$, the subgroup of $H$ preserving $V_{ijk}$ has to be equal to $G_{ijk}$ according to Part 1. In particular, for each pair $i< j$, the group $H$ contains the orthogonal subgroup $G_{ij}< G=SO(n)$. This is what I will be using. Since $H$ is a closed subgroup of $G$, it is a Lie subgroup, let ${\mathfrak h}$ denote its Lie algebra, regarded as a subalgebra of $o(n)$. The algebra  ${\mathfrak h}$ contains for each pair $i< j$ the commutative Lie subalgebra ${\mathfrak h}_{ij}$ equal to the Lie algebra of $G_{ij}$. In terms of matrices, ${\mathfrak h}_{ij}$ is generated by the "elementary skew-symmetric matrix" $E^{ij}$, where $E^{ij}$ has zero entries everywhere except at the positions $(ij), (ji)$:
$$
E^{ij}_{ij}= 1, E^{ij}_{ji}= -1.
$$
For instance,
$$
E^{12}= \left[\begin{array}{ccccc}
0&1&0&\ldots&0\\
-1&0&0&\ldots&0\\
0&0&0&\ldots&0\\
\vdots & \vdots &\vdots &\vdots &\vdots\\
0&0&0&\ldots&0\end{array}\right].
$$
The matrices $E^{ij}, 1\le i<j\le n$, span the Lie algebra $o(n)$ as a vector space since $o(n)$ consists of skew-symmetric $n\times n$ real matrices:
Each $M\in o(n)$ has entries $m_{ij}=-m_{ji}$ and, thus,
$$
M= \sum_{i<j} m_{ij} E_{ij}.
$$
Since $E^{ij}\in {\mathfrak h}$ for all $i<j$, it follows that
${\mathfrak h}= o(n)$ and, therefore, $H=SO(n)$. Theorem follows. qed

I can now finish the proof. Let $A\subset {\mathbb R}^n$ be an open subset as above. The subgroup $H$ of $SO(n)$ preserving $A$ has to equal $SO(n)$ by the theorem. Thus, for each $a\in A$, the domain $A$ contains the round sphere $S_a(0)$ centered at $0$ and of the radius $|a|$. Since $A$ is assumed to be open and connected, the set of radii $|a|$ as above, is an open connected subset $I$ in $[0,\infty)$. If $I$ has the form $[0, R)$, $R<\infty$, then $A$ is the open round ball of the radius $R$. If $I$ has the form $(r, R)$ then $A$ is a spherical shell. There are two more cases when $I$ is unbounded, corresponding to $A={\mathbb R}^n$ (if $I=[0,\infty)$) or
$$
A= \{x: r<|x|\}
$$
(if $I=(r,\infty)$). qed
Remark 1. In my answer I assumed that by a "rotation" you mean an element of $SO(n)$, i.e. a Euclidean rotation fixing the origin. If you allow more general rotations, the proof still goes through assuming that $A$ is bounded (and you made this assumption). Namely, if $H$ denotes the subgroup of $SE(n)$ (orientation-preserving Euclidean isometries) preserving $A$, then $H$ has to be a closed and bounded subgroup of $SE(n)$, hence, $H$ is compact. Every compact subgroup of $SE(n)$ fixes a  point in ${\mathbb R}^n$. (To find a fixed point, take for instance the Chebyshev center of any orbit $Hv\subset {\mathbb R}^n$.)  Changing the Cartesian coordinates so that this fixed point is the origin, reduces the problem to the one where $H\le SO(n)$. With more work, one can also handle the case of unbounded domains but I will not do this.
Remark 2. With a bit more work, one can prove that it suffices for $A$ to have an 8-fold rotational symmetry for each coordinate plane $V_{i,i+1}$, $i=1,...,n-1$. The proof is similar but one argues that the matrices $E^{i,i+1}$ generate $o(n)$ as a Lie algebra.
