Fundamental representation of $\mathfrak{so}(3)$ from the highest weight method

Question

Applying the definition of a Lie algebra $\mathfrak{g}$ corresponding to a Lie group $G$ as the tangent space at the identity $T_e(G)$, one can easily construct the Lie algebra $\mathfrak{so}(3)$ corresponding to $SO(3)$ as

$$ \mathfrak{so}(3) = \{ X \in \mathrm{GL}_3(\mathbb{R}):X^T=-X \} $$

i.e. it is the set of $3 \times 3$, real, anti-symmetric matrices. An obvious basis of this space is given by

$$ L_1 = \begin{pmatrix} 0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}, \quad L_2 = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ -1 & 0 & 0 \end{pmatrix}, \quad L_3= \begin{pmatrix} 0 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & -1 & 0 \end{pmatrix}. $$

Now, if I was to use Cartan's classification and use the highest weight method, I would write down the basis of my fundamental representation as

$$ L_x = \frac{1}{\sqrt{2}} \begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix}, \quad L_y =\frac{1}{\sqrt{2}}\begin{pmatrix}0 & -i & 0 \\ i & 0 & -i \\ 0 & i & 0 \end{pmatrix}, \quad L_z= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & -1 \end{pmatrix}. $$

These matrices are neither real nor anti-symmetric. I am aware that the highest weight method produces representations of the Lie algebra, but for the fundamental representation I would expect this to be the trivial case of $ d:\mathfrak{so}(3) \rightarrow \mathrm{GL}(V)$ where $d(X) = X$ for all $X \in \mathfrak{so}(3)$, in other words, I'd expect these matrices to agree, or at least be related by a similarity transformation. As these matrices operate on Euclidean space, there can't be a real similarity transformation to relate these two representations.

My question

Why does the fundamental representation produced by the highest weight method give us a representation that is complex which is neither anti-symmetric or real as the elements of $\mathfrak{so}(3)$ should be?

Answer 1

First of all, just for clarity of future readers: the terminology "fundamental representation" is used a bit differently in physics than in mathematics (cf. 1, 2, 3). The representation we are looking at here is the irreducible one of spin $1$ i.e. highest weight $2$; most mathematicians would not count that among what they call "fundamental representations" (for them, the algebra in question has only one fundamental rep, the one with highest weight $1$, which is of dimension $2$). Instead, they might call it the "defining" representation of $\mathfrak{so}_\color{red}3$ (namely, on a $\color{red}3$-dimensional vector space).

The next big obstacle is that further, mathematicians' and physicists' conventions on (in particular compact) Lie algebras differ by multiplying with the imaginary unit $i$. Because I am a mathematician, I'll stick with that notation, differing from the one in the Wikipedia article you cite; meaning that for me,

$$ L_x = \frac{1}{\sqrt{2}} \begin{pmatrix} 0 & i & 0 \\ i & 0 & i \\ 0 & i & 0 \end{pmatrix}, \quad L_y =\frac{1}{\sqrt{2}}\begin{pmatrix}0 & 1 & 0 \\ -1 & 0 & 1 \\ 0 & -1 & 0 \end{pmatrix}, \quad L_z= \begin{pmatrix} i & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & -i \end{pmatrix}. $$

Actually, this is necessary, because when you define $\mathfrak{so}_3$ as the anti-symmetric real matrices, you are already following mathematical convention, so we have to stay consistent.

Now even if you grant me that, on first sight your question remains: These are neither real nor anti-symmetric. (They are however anti-hermitian, and make me very happy because comparing some computations I did a long time ago on a bus to this and the higher dimension analogues in that WP article makes me realise that, up to multiplying through with $i$, I did some quantum mechanics without knowing it.)

But now it's conjugation and scaling: With $A:=\dfrac{1}{\sqrt2}\pmatrix{1&0&i\\0&-\sqrt2 i&0\\1&0&-i}$ and $s(M):=AMA^{-1}$, we have

$$s(L_1)= L_x, s(L_3)= L_y, s(L_2)=-L_z . $$

(How to come up with the above base change matrix: Similar to the computations in step 2 in my answer to to Explicit isomorphism between the four dimensional orthogonal Lie algebra and the direct sum of special linear Lie algebras of dimension 3.)

It is not too surprising that the "right" general matrices that come from highest weight theory, i.e. the $L_x, L_y, L_z$ above (in mathematical or physical convention) have complex entries, because the theory relies on complex representations of the complexified Lie algebra, in this case $\mathbb C \otimes \mathfrak{so}_3 \simeq \mathfrak{sl}_2(\mathbb{C})$. What is special for this representation is that there is a similarity transform (the $s$, or rather its inverse, above) which makes it "entirely real" i.e. gives out matrices with all real entries. Of course this would be expected in dimension $3$ because after all we can define $\mathfrak{so}_3$ as a certain set of $3\times3$-matrices with all real entries (and that's what motivated your question). But if instead we had written that Lie algebra as $\mathfrak{su}_2$, that would have been surprising: The standard way one defines $\mathfrak{su}_2$ involves complex $2\times 2$-matrices, and there is no way to transform those into ones with only real entries. If I recall correctly, that parity distinction repeats in higher dimensions, i.e. the irreducible representations in even dimensions (for physicists: half-integer spin) are "truly complex" (or actually, "quaternionic" or "hermitian" or "pseudoreal", depending on what terminology your source likes), whereas the ones in odd dimensions (for physicists: integer spin) can indeed be written with real (or I guess in physicists' notation: purely imaginary) coefficients.