Is there a reliable method for finding irreducible subrepresentations in $GL(n,\mathbb R)$? Specifically, this question is related to this question which I asked last night. So I will draw my examples from the Cyclic Group, $\mathbb Z_n$.
Given a group representation in the real-valued General Linear Group, $GL(n,\mathbb R)$, is there a systematic method for block-diagonalizing it into a direct sum of irreducible subrepresentations?
From the question I asked last night, I got a sense for how to do this for $n\times n$ representations of $\mathbb Z_n$ when $n=2, 3, 4, \text{ and } 6$, but I'm completely at a loss for how to reduce such a representation when $n$ is any larger (like 7, 8, or 9) or odd (especially 5).
Specifically, I've been working on reducing the representation of $\mathbb Z_5$ given by the cyclic column permutations of the $5\times 5$ identity matrix. Attempting to reduce it into 1 subrepresentation of dimension 1 and 2 subrepresentations of dimension 2 has not gotten me anywhere. Because the order is odd, it seems impossible to impose cyclic conditions on the components of the basis vectors - meaning that no matter how I choose the first 4 vectors, there is no fifth vector that works.
If it makes answering the question any easier, feel free to restrict discussions to representations consisting entirely of orthogonal matrices since the representations must obviously be unitary and real.
 A: If I have it right, it sounds like you're trying to take a particular real matrix representation for a finite group and decompose it into (real) irreducibles.  My approach would be to 1) make a character table for the group 2) calculate the Schur indicator to see how the complex irreducibles pair to create real representations 3) use the characters to create projection matrices onto isotopic components (and get bases for each isotypic component) 4) use a little ingenuity to split the isotopic components into irreducibles, if needed.
For $C_5$ in particular: let $t$ be the generator and $\alpha=e^{2\pi i/5}$.  The five irreducible representations are all 1D and given by $\rho_j(t^n)=\alpha^{j n}$ for $j=0,1,2,3,4$.  The Schur indicator for $\rho_j$ is
$$\frac{1}{5}\sum_{n=0}^4 (\alpha^{j n})^2$$
If $j=0$, this is $1$, indicating that the trivial representation is a real representation.
The other $j$ are coprime to $5$, and since $2$ is also coprime to $5$, it turns out we can reparameterize:
$$\frac{1}{5}\sum_{n=0}^4\alpha^n=0$$
hence the other representations are complex.  This means the conjugate pairs $\rho_j$ and $\rho_{5-j}$ pair up to form 2d irreducible real representations.
So, $C_5$ has three distinct irreducible real representations:


*

*The trivial representation with character $\chi_0(t^n)=1$.

*For $j=1,2$, the 2D representation with character $$\chi_j(t^n)=\alpha^{jn}+\alpha^{-jn}=2\cos(2\pi j n/5)$$
Your particular representation in $\mathbb{R}^5$ of $C_5$ by cyclic permutation has the character $\chi$ with $\chi(1)=5$ and $\chi(t^n)=0$ for $n=1,2,3,4$.  Using the (real) character product, we have
\begin{align}
(\chi,\chi_0)&=\frac{1}{5}\sum_{n=0}^4\chi(t^n)\chi_0(t^n)=\chi_0(1)=1\\
(\chi,\chi_1)&=\frac{1}{5}\sum_{n=0}^4\chi(t^n)\chi_1(t^n)=\chi_1(1)=2\\
(\chi,\chi_2)&=\frac{1}{5}\sum_{n=0}^4\chi(t^n)\chi_1(t^n)=\chi_2(1)=2
\end{align}
This indicates that $\chi=\chi_0+\chi_1+\chi_2$.
Let $\rho$ denote your representation of $C_5$ on $\mathbb{R}^5$ by cyclic permutation, where in particular we have $\rho(t^i)e_j=e_{i+j}$ on the standard basis vectors, with the indices modulo $5$.
Up to a scale multiple, the (real) projector associated to $\chi_j$ is
$$\pi_j(v)=\sum_{n=0}^4 \chi_j(t^n)\rho(t^n)v.$$
(You can make it so it doesn't scale vectors, but I didn't work it out here.)  From here, you basically just calculate the column space of each projection matrix to find the bases for each irreducible component, and then the matrices of the original representation will be block diagonal.
For $\pi_0$, its image is sort of obviously the span of $(1,1,1,1,1)$.
For $\pi_1$, I get the span of $(1,0,-1,(1-\sqrt{5})/2,(-1+\sqrt{5})/2)$ and $(0,1,(-1+\sqrt{5})/2,(1-\sqrt{5})/2,-1)$.
For $\pi_2$, I get the span of $(1,0,-1,(1+\sqrt{5})/2,(-1-\sqrt{5})/2)$ and $(0,1,(-1-\sqrt{5})/2,(1+\sqrt{5})/2,-1)$ (the Galois conjugate).
With respect to this basis, I get that the generator $t$, which is as you described in your other post, has the matrix
$$\begin{pmatrix}1\\&(-1+\sqrt{5})/2&-1\\&1&0\\&&&(-1-\sqrt{5})/2&-1\\&&&1&0\end{pmatrix}$$
If I had bothered to make each 2D basis orthogonal, we would be seeing the standard 2D representations of $C_5$ counter-rotating.

The exercise for $S_n$ isn't so different (for fixed $n$).  However, don't get the impression that finding the irreducible real representations of $S_n$ will solve the problem for subgroups of $S_n$.  You will have to figure out how the restricted representations split.
One difference is that all the irreducible representations of $S_n$ are absolutely irreducible, meaning, at the least, they are all real.
The $n$-dimensional standard representation of $S_n$ splits into two irreducibles.  This is an exercise of character theory.  It is also fairly easy to show the $(n-1)$-dimensional subrepresentation is irreducible by demonstrating that every non-zero vector generates it.
A: I've done quite a bit of research since asking this question. Here is what I've found.
1) This question is quite broad, but from Cayley's Theorem, it is enough to just answer this question as it applies to the Permutation Group (or Symmetric Group), $S_n$.
2) Answering this question conclusively for $S_n$ is quite challenging, so it may be worth-while seeing how it goes with subgroups - such as the Cyclic Group, $C_n$, and the Dihedral Group, $D_n$. Since $D_n$ can always be constructed from $C_n$ by tensoring it with the alternating representation (or sign representation - $(1,-1,1,-1,1,...)$) of $S_n$, I'll stick exclusively to $C_n$.
3) Obviously, since we're discussing representations over $\mathbb R$, a lot of important theorems don't apply. I have a few conclusions that I suspect are correct but can't prove. I'll point them out when they happen.
4) I've found a paper that I think answers this question, but honestly, it's a bit over my head. A user-friendly summary of what they're talking about would be greatly appreciated.
First of all, I want to plug "The orthogonal and the natural representation for symmetric groups" one more time. I read through the article, and it seems to be saying that finding real-valued representations of a generic permutation is incredibly challenging. It provides an explanation of a process for finding such representations "efficiently," but I'm having a lot of trouble following the explanation and it appears to be addressing individual permutations rather than the entire group.
Next, the "natural representation" of $S_n$ is real-valued and (in some bases) so is the "standard representation." On top of that, both representations can live in the Orthogonal Group, $O(n)$, which gives us some hope since that is a pretty well-behaved subgroup of $GL(n,\mathbb R)$.
Here's the upshot: There is a fairly straight-forward method for block-diagonalizing $C_n$ into its irreducible representations (which I will explain shortly). This method produces a basis which, when applied to the rest of $S_n$, block-diagonalizes it into a $1\times 1$ and an $(n-1)\times(n-1)$ direct sum. I suspect (but cannot prove) that the $(n-1)\times(n-1)$ submatrix is a version of the irreducible standard representation.
Here is how to find the basis which block-diagonalizes $C_n$ into irreducible subrepresentations (and maybe $S_n$ too):
All elements of $C_n$ drawn from the natural representation of $S_n$ (besides the identity) have zeros on the diagonal. That means they all have the same complex eigenvalues - namely, complex roots of unity of order $n$.
Consider the argument of each root in the upper half plane, and construct a $2\times 2$ rotation matrix for that angle. Construct a direct sum of each of the $1\times 1$ (real roots) and $2\times 2$ (complex roots) matrices.
Now you have your group element, $G$, and its block diagonal, $B$. You want to construct the basis which transforms from one to the other. In other words, we want the columns of the matrix $P$ such that
$$P^{-1}GP=B
$$
or
$$GP=PB
$$
With a little elbow grease and some old-fashioned linear algebra, you can determine that for the root $1$ the basis vector is $(1, 1, 1,...)$. For $-1$, it's $(1,-1,1,-1,...)$, and for any given argument $\theta$ the two basis vectors are
$$\pmatrix{1\\ \cos((n-1)\theta)+\sin((n-1)\theta)\\ ... \\ \cos(2\theta)+\sin(2\theta)\\\cos(\theta)+\sin(\theta)}\text{and}\pmatrix{1\\ \cos((n-1)\theta)-\sin((n-1)\theta)\\ ... \\ \cos(2\theta)-\sin(2\theta)\\\cos(\theta)-\sin(\theta)}
$$
From the original question, here is transition matrix $P$ for $C_5$:
$$\pmatrix{1&1&1&1&1\\1&\cos(\frac{8\pi}{5})+\sin(\frac{8\pi}{5})& \cos(\frac{8\pi}{5})-\sin(\frac{8\pi}{5})&\cos(\frac{16\pi}{5})+\sin(\frac{16\pi}{5})&\cos(\frac{16\pi}{5})-\sin(\frac{16\pi}{5})\\1&\cos(\frac{6\pi}{5})+\sin(\frac{6\pi}{5})& \cos(\frac{6\pi}{5})-\sin(\frac{6\pi}{5})&\cos(\frac{12\pi}{5})+\sin(\frac{12\pi}{5})&\cos(\frac{12\pi}{5})-\sin(\frac{12\pi}{5})\\1&\cos(\frac{4\pi}{5})+\sin(\frac{4\pi}{5})& \cos(\frac{4\pi}{5})-\sin(\frac{4\pi}{5})&\cos(\frac{8\pi}{5})+\sin(\frac{8\pi}{5})&\cos(\frac{8\pi}{5})-\sin(\frac{8\pi}{5})\\1&\cos(\frac{2\pi}{5})+\sin(\frac{2\pi}{5})& \cos(\frac{2\pi}{5})-\sin(\frac{2\pi}{5})&\cos(\frac{4\pi}{5})+\sin(\frac{4\pi}{5})&\cos(\frac{4\pi}{5})-\sin(\frac{4\pi}{5})}
$$
Any observations about the form of $S_n$ in such a basis are welcome (particularly if they include an up-or-down conclusion about whether the $4\times 4$ submatrix is in fact the standard representation).
