Polynomials invariant under the action of SO(3) Let $SU(2)$ act on $V_2=\{ax^2+bxy+cy^2:a,b,c\in\mathbb C\}$ by $$\begin{pmatrix}a&b\\c&d\end{pmatrix}f(x,y)=f(ax+cy,bx+dy).$$ Then $SU(2)$ also acts on the $n$-th symmetric power $S^n V_2$. Show that $(S^n V_2)^{SU(2)}$ and $k_n[X,Y,Z]^{SO(3)}$ have the same dimension, where $k_n[X,Y,Z]$ is the space of degree n homogeneous polynomials.
Notations: $(S^n V_2)^{SU(2)}$ is the space of elements of $S^n V_2$ invariant under the action of $SU(2)$,
$k_n[X,Y,Z]^{SO(3)}$ is the space of elements in $k_n[X,Y,Z]$ invariant under the action of $SO(3)$, which is given by $\begin{pmatrix}a&b&c\\d&e&f\\g&h&i\end{pmatrix}f(X,Y,Z)=f(aX+dY+gZ,...).$
My attempt: I know that $SU(2)/\{\pm I\}\cong SO(3)$. Also, $-I$ acts trivially on $V_2$, so $SO(3)$ acts on $V_2$. I guess I need to use this to show $k_n[X,Y,Z]^{SO(3)}$ is isomorphic to $(S^n V_2)^{SU(2)}$. However, I am not sure how to prove this. Can anyone help?
 A: We can replace the group $SU(2)$ with $SL_2$ for this question as $SU(2)$ is the maximal compact subgroup of $SL_2$, so the vector space of invariants does not change. Moreover, we can also replace $V_2$ with $V=\mathfrak{sl}_2$, the Lie algebra of $SL_2$, since $V_2$ and $\mathfrak{sl}_2$ are isomorphic as $SL_2$ modules.
Note that the action of $SL_2$ on $V$ is exactly how the map $SL_2\rightarrow SO(3)$ is achieved: $SL_2$ acts on the three dimensional vector space $V$, so it gives a map $SL_2\rightarrow GL_3$. Moreover, $SL_2$ preserves the Killing form of $V=\mathfrak{sl}_2$, hence the image lies in $SO(3)$. You already mentioned that this map is surjective.
The claim immediately follows: Note that $\mathbb{C}_n[X,Y,Z]$ and $S^n(V^*)$ are isomorphic, since $V$ is three dimensional and $S^n(V^*)$ is isomorphic to the space of polynomials of degree $n$ on $V$. Moreover, $SL_2$ modules are self-dual; thus, $S^n(V^*)=S^n(V)$.
Since $SL_2$ acts on $V$ "just like" $SO(3)$, the representations $SL_2$ on $S^n(V)$ and $SO(3)$ on $\mathbb{C}_n[X,Y,Z]$ are the same.
I know the proof is a bit loose, feel free to ask anything.
A: I will give a very rough sketch of how I think this can be proven.
To establish the isomorphism between invariants, we could try to establish the isomorphisms of the representations themselves.
As a first step, we need to use the correspondence between $n$-symmetric tensors and degree-$n$ homogeneous polynomials. We can define this correspondence by defining it between the separable tensors and monomials. Pick a basis for $V_2$, say $\{v_1, v_2, v_3\}$, then we can map a tensor of the form $\operatorname{Sym}(v_{i_1}\otimes \dots \otimes v_{i_n})$ to a monomial $X^a Y^b Z^c$, where $a, b, c$ count the number of times $v_1, v_2, v_3$ appears in the tensor, respectively. Through this correspondence, we can interpret $k_n[X, Y, Z]$ as the $n$-th symmetric power of the $3$-dimensional space of linear homogeneous polynomials in $X, Y, Z$, namely $k_1[X, Y, Z]$. It shouldn't be too hard to show that this correspondence is equivariant. It would probably follow from working through how the representation behaves on a symmetric power.
Next we may note that symmetric powers are functors, so we need only show that the representations $V_2$ (technically of $SU(2)/\{\pm I\}$) and $k_1[X,Y,Z]$ are isomrophic. Since these are just the three dimensional irreps of the two groups (or at least the right action versions), this is a standard result.

This proof idea sort of dodges the question of "what are the invariants". For $SO(3)$, this turns out not so hard to answer. Invariant polynomials of $SO(3)$ are all of the form $\phi(X^2+Y^2+Z^2)$ for some polynomial $\phi$, which means that $k_n[X,Y,Z]^{SO(3)} = \langle (X^2+Y^2+Z^2)^{n/2} \rangle$ when $n$ is even, and trivial otherwise. For $SU(2)$, describing symmetric polynomials are a bit trickier, but you can decompose $S^n V_2$ into irreps, and as it turns out, when $n$ is even you get exactly one invariant.
A: Question: "However, I am not sure how to prove this. Can anyone help?"
Answer: @user14411 - If $V:=\mathbb{C}\{e_1,e_2\}$ and $V^*:=\mathbb{C}\{x_1,x_2\}$ and if $V_2 \cong Sym^2(V^*)$ you would get
$$F1.\text{  }Sym^n(V_2) \cong Sym^n(Sym^2(V^*)),$$
and since $G:=SU(2)$ is semi simple, there is a decomposition
$$P1.\text{  } Sym^n(Sym^2(V)) \cong \oplus_{\lambda} V(\lambda)$$
with $V(\lambda)$ an irreducible $G$-module. There is an equality
$$Sym^n(Sym^2(V))^{G} \cong \oplus_{\lambda} V(\lambda)^G$$
and I believe there are "lists" giving formulas for the dimension $dim(V(\lambda)^G)$
for such $\lambda$. If $F1$ holds and if you know the decomposition in $P1$ you get a formula. Much effort has been put into constructing the invariants $V(\lambda)^G$ and into calculating such dimensions.
Similarly: The vector space $k_n[x,y,z] \cong Sym^n(k\{x,y,z\})$ is the symmetric power of the dual $W^*$ of the standard representation $W$ of $SO(3)$, and I believe there are lists giving  generators and dimensions for $Sym^n(W^*)^{SO(3)}$. Look up "invariant theory".
In general if $\mathbb{S}_{\lambda}, \mathbb{S}_{\mu}$ are Schur functors and $V$ is a finite dimensional $G$-module with $G$ a semi simple algebraic group, it follows there is a direct sum decomposition into irreducible $G$-modules $W_i$:
$$ \mathbb{S}_{\lambda}(\mathbb{S}_{\mu}(V)) \cong \oplus_i W_i.$$
And if you seek $\mathbb{S}_{\lambda}(\mathbb{S}_{\mu}(V))^G$ you should first calculate $W_i^G$.
