Rotation invariant bilinear form There is a passage in a book I am reading that I don't understand, regarding invariance under group transformations.
The author starts by saying the functions $\{\phi_\alpha\}$, with $\alpha\in\{-1,0,1\}$, span the triplet representation of $SO(3)$. This means that under rotations they transform as $\phi'_\alpha=\sum_\beta D_{\alpha\beta}\phi_\beta$, where $D$ are usual rotation matrices.
This I think I understand and the picture in my mind is just three ortogonal vectors (is this appropriate?)
Then the author says, without further explanations, that the bilinear form $\sum_\alpha \phi_\alpha\phi_{-\alpha}(-1)^{\alpha-1}$ is invariant under rotations.
Now I am completely lost, because I thought the invariant bilinear form should be the usual dot product, $\sum_\alpha \phi_\alpha^2$.
Can someone clarify this, please?
 A: If the functions $\phi_\alpha$ span an irreducible representation, they must be linear combinations of spherical harmonics $Y_{l,m}$. Let us assume they are exactly the spherical harmonics, for simplicity. For the triplet index $\alpha$ is $m$ and $l=1$.
What your book is telling you is that $$\sum_{m=-1}^1 Y_{1,m}Y_{1,-m}(-1)^m$$ should be rotation invariant, i.e. a scalar.
This is true and it boils down to two well known properties of spherical harmonics. First, $Y_{l,-m}=(-1)^mY_{l,m}^*$, so that your expression becomes $\sum_{m=-1}^1 Y_{1,m}Y_{1,m}^*$. Second, the addition theorem,
$$ \sum_{m=-l}^l Y_{l,m}(\vec x)Y_{l,m}^*(\vec y)=\frac{2l+1}{4\pi}P_l(\vec x \cdot \vec y),$$ where $P_l$ are Legendre polynomials.
A: Invariance of a form doesn't imply that its the dot product. Let $V$ be a vector space and < , > a bilinear form on it. We know that if any basis is given to us, we can represent the form as a matrix ($A$) and that under a basis change (say by $P$), the matrix of the the form changes as $P^tAP$.
Now, had the form initially been a dot product, i.e to say $A=I$, it would have changed under the basis change (by $P$) to $P^tIP=P^tP$. If P were an orthogonal matrix, $P^tP=I$ would have once again yielded me $A=I$. So, though we changed our basis, the matrix of the form remained the same and hence we would call the dot product (a particular bilinear form) to be invariant under an orthogonal change of basis.
However for any other arbitrary bilinear form, invariance would mean that if the original matrix of form was A, under basis change by $P$, it should become $P^tAP=A$. Which in the light of the question would mean that the rotation matrices $D$ will satisfy $D^tAD=A$ if A was the matrix of your bilinear form. Try with this understanding. (Can you enlighten me about $\beta$ in the question?)
