Discrete non-Abelian subgroups and invariants in a unitary group

Let $$G=U(3),$$ be the unitary group. Here we consider $G$ in terms of the fundamental representation of U(3). Namely, all of $g \in G$ can be written as a rank-3 (3 by 3) matrices.

1. Can we find some subgroup of Lie group, $$k \in K \subset G= U(3)$$ such that

$$k^T \{P_1, P_2, P_3, -P_1, - P_2, - P_3 \} k =\{P_1, P_2, P_3, -P_1, - P_2, - P_3\} .$$ This means that set $\{P_1, P_2, P_3, -P_1, - P_2, - P_3\}$ is invariant under the transformation by $k$. Here $k^T$ is the transpose of $k$. What is the full subset (or subgroup) of $K$?

Here we define: $$P_1 = \left( \begin{array}{ccc} 0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} \right),\;\;\;\; P_2 = \left( \begin{array}{ccc} 0 & 0 & -1 \\ 0 & 0 & 0 \\ 1 & 0 & 0 \\ \end{array} \right),\;\;\;\; P_3 = \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & -1 & 0 \\ \end{array} \right).$$

This means that $k^T P_a k= \pm P_b$ which may transform $a$ to a different value $b$, where $a,b \in \{1,2,3 \}$. But overall the full set $\{P_1, P_2, P_3, -P_1, - P_2, - P_3\}$ is invariant under the transformation by $k$.

There must be a trivial element $k=$ the rank-3 identity matrix. But what else can it allow? In particular, I can see a symmetric group S$_4$ and $\mathbb{Z}_3$ and a possible additional structure in $K$.

How could we determine the complete $K$?

• I believe that you mean that the full set $\{P1,P2,P3,−P1,−P2,−P3\}$ is one-to-one (injective and surjective) mapping to $k^T \{P1,P2,P3,−P1,−P2,−P3\} k$ for all $k \in K \subset G$. – wonderich Apr 12 '18 at 5:46

In the OP's previous question I already proposed an answer as follows:

The subgroup $K=\{k\in U(3)\mid k^TP_1k=\pm P_1\}$ of $U(3)$ is isomorphic to $(SU(2)\rtimes\mathbb{Z}_2)\times U(1)$ where $SU(2)\rtimes\mathbb{Z}_2$ is the group of $2\times2$ unitary matrices with determinant $\pm1$.

In particular, we have a specific matrix form \begin{align*} K &= \Bigl\{ \begin{pmatrix} \alpha & \mp\bar\beta & 0 \\ \beta & \pm\bar\alpha & 0 \\ 0 & 0 & \gamma \end{pmatrix} \in U(3) \mid \alpha,\beta,\gamma\in\mathbb{C},\,|\alpha|^2+|\beta|^2=|\gamma|=1 \Bigr\} \\ &= \Bigl\{ \begin{pmatrix} A & \mathbf{0} \\ \mathbf{0} & \gamma \end{pmatrix} \in U(3) \mid A\in U(2),\,\det A=\pm1,\,|\gamma|=1 \Bigr\} \tag{*} \end{align*}

At first we fix a permutation matrix $Q=\begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}$.

Note that the order of $Q$ is $3$, and $Q^2=Q^{-1}=Q^T$. Moreover we have $$QP_1Q^T=P_2 \quad\text{and}\quad Q^2P_1(Q^2)^T=Q^TP_1Q=P_3$$

Claim. The set of matrices $\{\bar k\in U(3) \mid \bar k^TP_1 \bar k=\pm P_2\}$ is $KQ^T=\{kQ^T\mid k\in K\}$.

Proof. If $k\in U(3)$ satisfies $k^TP_1k=\pm P_1$, then $k\in K$ (see the equation ($*$) above), and $$(kQ^T)^TP_1(kQ^T)=Q(k^TP_1k)Q^T=\pm(QP_1Q^T)=\pm P_2$$ On the contrary, if $\bar k\in U(3)$ satisfies $\bar k^TP_1\bar k=\pm P_2$, then $$\bar k^TP_1\bar k=\pm P_2=\pm(QP_1Q^T) \implies (\bar kQ)^TP_1(\bar kQ)=\pm P_1$$ so that $\bar kQ=k \Rightarrow \bar k=kQ^T$ for some $k\in K$.

In a similar way, it is routine to check that \begin{align*} K_{00} \equiv \{k\in U(3) \mid k^TP_1 k=\pm P_1\} &= Q^0KQ^0 = K \\ K_{02} \equiv \{k\in U(3) \mid k^TP_1 k=\pm P_2\} &= Q^0KQ^2 = KQ^T \\ K_{01} \equiv \{k\in U(3) \mid k^TP_1 k=\pm P_3\} &= Q^0KQ^1 = KQ \\ K_{10} \equiv \{k\in U(3) \mid k^TP_2 k=\pm P_1\} &= Q^1KQ^0 = QK \\ K_{12} \equiv \{k\in U(3) \mid k^TP_2 k=\pm P_2\} &= Q^1KQ^2 = QKQ^T \\ K_{11} \equiv \{k\in U(3) \mid k^TP_2 k=\pm P_3\} &= Q^1KQ^1 = QKQ \\ K_{20} \equiv \{k\in U(3) \mid k^TP_3 k=\pm P_1\} &= Q^2KQ^0 = Q^TK \\ K_{22} \equiv \{k\in U(3) \mid k^TP_3 k=\pm P_2\} &= Q^2KQ^2 = Q^TKQ^T \\ K_{21} \equiv \{k\in U(3) \mid k^TP_3 k=\pm P_3\} &= Q^2KQ^1 = Q^TKQ \end{align*}

Now let us find $\{\pm P_1,\pm P_2,\pm P_3\}$ invariant matrices. There are three cases $K_{00},K_{02},K_{01}$ according to $k^TP_1k=\pm P_1$, $\pm P_2$ or $\pm P_3$.

Case 1. Find the intersection of $K_{00}$ with $K_{10},K_{12},K_{11}$ according to the type of $k^TP_2k$. \begin{align*} K_{00}\cap K_{10} &= K \cap QK = \varnothing \\ K_{00}\cap K_{12} &= K \cap QKQ^T = \bigl\{ \begin{pmatrix} \alpha & 0 & 0 \\ 0 & \beta & 0 \\ 0 & 0 & \gamma \end{pmatrix} \text{ where \alpha\beta=\pm1, \alpha\gamma=\pm1} \bigr\} \\ K_{00}\cap K_{11} &= K \cap QKQ = \bigl\{ \begin{pmatrix} 0 & \alpha & 0 \\ \beta & 0 & 0 \\ 0 & 0 & \gamma \end{pmatrix} \text{ where \alpha\beta=\pm1, \alpha\gamma=\pm1} \bigr\} \end{align*}

Now find the intersection with the remaining case $K_{20},K_{22},K_{21}$. \begin{align*} K_{00}\cap K_{12}\cap K_{20} &= K \cap QKQ^T \cap Q^TK= \varnothing \\ K_{00}\cap K_{12}\cap K_{22} &= K \cap QKQ^T \cap Q^TKQ^T = \varnothing \\ K_{00}\cap K_{12}\cap K_{21} &= K \cap QKQ^T \cap Q^TKQ \\ &= \bigl\{ \begin{pmatrix} \alpha & 0 & 0 \\ 0 & \beta & 0 \\ 0 & 0 & \gamma \end{pmatrix} \text{ where \alpha\beta=\pm1, \alpha\gamma=\pm1, \beta\gamma=\pm1} \bigr\} \\ K_{00}\cap K_{11}\cap K_{20} &= K \cap QKQ \cap Q^TK = \varnothing \\ K_{00}\cap K_{11}\cap K_{22} &= K \cap QKQ \cap Q^TKQ^T \\ &= \bigl\{ \begin{pmatrix} 0 & \alpha & 0 \\ \beta & 0 & 0 \\ 0 & 0 & \gamma \end{pmatrix} \text{ where \alpha\beta=\pm1, \alpha\gamma=\pm1, \beta\gamma=\pm1} \bigr\} \\ K_{00}\cap K_{11}\cap K_{21} &= K \cap QKQ \cap Q^TKQ = \varnothing \end{align*}

We can find intersections for the other cases $K_{02}$ and $K_{01}$ in a similar way.

Finally, we have the following $16\times 6$ invariant matrices:

For $\alpha,\beta,\gamma\in\mathbb{C}$ satisfying $\alpha\beta=\pm1$, $\alpha\gamma=\pm1$, and $\beta\gamma=\pm1$, \begin{align*} K_{00}\cap K_{12}\cap K_{21} &\Rightarrow \begin{pmatrix} \alpha & 0 & 0 \\ 0 & \beta & 0 \\ 0 & 0 & \gamma \end{pmatrix} \quad K_{00}\cap K_{11}\cap K_{22} \Rightarrow \begin{pmatrix} 0 & \alpha & 0 \\ \beta & 0 & 0 \\ 0 & 0 & \gamma \end{pmatrix} \\ K_{02}\cap K_{10}\cap K_{21} &\Rightarrow\begin{pmatrix} \alpha & 0 & 0 \\ 0 & 0 & \beta \\ 0 & \gamma & 0 \end{pmatrix} \quad K_{02}\cap K_{11}\cap K_{20} \Rightarrow\begin{pmatrix} 0 & 0 & \alpha \\ \beta & 0 & 0 \\ 0 & \gamma & 0 \end{pmatrix} \\ K_{01}\cap K_{10}\cap K_{22} &\Rightarrow\begin{pmatrix} 0 & \alpha & 0 \\ 0 & 0 & \beta \\ \gamma & 0 & 0 \end{pmatrix} \quad K_{01}\cap K_{12}\cap K_{20} \Rightarrow\begin{pmatrix} 0 & 0 & \alpha \\ 0 & \beta & 0 \\ \gamma & 0 & 0 \end{pmatrix} \end{align*}

Note that there are 16 solutions satisfying $\alpha\beta=\pm1$, $\alpha\gamma=\pm1$, $\beta\gamma=\pm1$, which are $\alpha,\beta,\gamma=\pm1$ (8 solutions of order 1 or 2) and $\alpha,\beta,\gamma=\pm i$ (8 solutions or order 4).

Therefore the subgroup of $U(3)$ containing invariant matrices are isomorphic to the finite group $$(\mathbb{Z}_2\times\mathbb{Z}_2\times\mathbb{Z}_4) \rtimes S_3 \cong \mathbb{Z}_4\times S_4 \cong\langle i\rangle\times D(2,3,4)$$ where $\langle i\rangle=\{\pm I,\pm iI\}\cong\mathbb{Z}_4$ and $D(2,3,4)$ is the von Dyck group which is isomorphic to $S_4$.

More specifically, $D(2,3,4)=\langle a,b,c \mid a^2=b^3=c^4=abc=I\rangle$ is represented in $U(3)$ as follows: $$a = \begin{pmatrix} -1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix}, \quad b = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}, \quad c = \begin{pmatrix} 0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix}$$

(The idea of this isomorphism was suggested by @wonderich.)

• @ ChoF, +1, thanks for your answer. It looks to me that, I can obtain a finite group of order 48 (at least) as the subgroup in the U(3), satisfying ${P1,P2,P3,−P1,−P2,−P3} = k^T{P1,P2,P3,−P1,−P2,−P3}k$ for all $k \in K\subset$ U(3). – annie marie heart Apr 12 '18 at 14:23
• @ ChoF, however, it looks to me that you suggest a larger group structure, that maps $k^T P_a k=P_b$. Does it guarantee that ${P1,P2,P3,−P1,−P2,−P3} = k^T{P1,P2,P3,−P1,−P2,−P3}k$ as a full set? Maybe the group is smaller than what you write? – annie marie heart Apr 12 '18 at 14:25
• @ ChoF, the SU(2) structure in $k_1 \in K_1$ that makes $k_1^T P_1k_1=\pm P_1$ is different from the $k_2 \in K_2$ that makes $k_2^T P_2k_2=\pm P_2$. Therefore, one needs to find intersection between them. My earlier answer is that all the continuous SU(2) may be gone after intersecting the set. – annie marie heart Apr 12 '18 at 19:39
• with this condition $\{P_1,P_2,P_3,−P_1,−P_2,−P_3\}=k^T\{P_1,P_2,P_3,−P_1,−P_2,−P_3\}k$. So the overall $K$ should not have a continuous group, yes? – annie marie heart Apr 12 '18 at 19:40
• @ ChoF, may I also understand your counting? It looks that if I consider $\alpha,\beta,\gamma$ as complex, with 6 matrices, there can be $$2^3 \times 6=48$$ discrete elements, agreeing with what I said in my first line. But I dont understand your $4^3 \times 6$ counting. – annie marie heart Apr 12 '18 at 21:03