Intuitive explanation for orthogonality relations between matrix elements of irreps It is well known that the matrix elements of complex irreducible representations of a finite group $G$ are orthogonal, i.e.
$$
\langle T_{ij}^\lambda, T_{kl}^{\rho} \rangle = \frac{1}{|G|} \sum_{g\in G} T_{ij}^\lambda(g) T_{kl}^\rho(g)^* = \frac{\delta_{ik} \delta_{jl} \delta_{\lambda \rho}}{d_\lambda},
$$
where $T_{ij}^\lambda$ is the $(i,j)$ matrix element of irreducible representation (irrep) indexed by $\lambda$, $d_\lambda$ is the dimension of this irrep, and $\delta_{\lambda \rho}$ means that $\lambda$ and $\rho$ correspond to an equivalent irrep.
The above can be strengthened, because in fact the matrix elements of irreps form a basis for the space of functions on $G$.
As a consequence, we can expand a function on $G$ as a linear combination of matrix elements, giving us some kind of generalization of classical Fourier theory. I want to explain this idea to an audience that knows classical Fourier theory but hardly anything about representations. My goal is to keep it high level and intuitive. I have looked at some proofs of the statement above, but they all seem fairly long, reliant on other results, and it's hard to distill the "essence" of why the above must be true. I would appreciate a short and intuitive explanation of the result above, that conveys the gist to an audience who does not yet know much representation theory.
 A: There are two important underlying facts:

*

*If you take the action of $G$ on $End(V_{\lambda}) $, this decomposes as $d_{\lambda}$ copies of $V$. You can think that $g$ acts on each column separately.

*The function

$$ A, B \mapsto \frac{1}{|G|} \sum_{g \in G} tr(gA) tr(gB)^*$$
gives an invariant pairing
$$ End(V_{\lambda}) \otimes_{\mathbb{C}} End(V_{\mu} ) \to \mathbb{C}$$
Given these two facts, you can see that for $\lambda \neq \mu$ this must be zero, because there is no invariant pairing between two different irreducible representations. In the case $\lambda = \mu$, you know that when restricted to a single colimn in both components it must be a multiple scalar of the unique invariant product $V_{\lambda} \otimes V_{\lambda} \to \mathbb{C} $, which is the standard hermitian product in some basis (the one in which $G$ acts in a unitary way). Here I think you can put under the carpet the computations showing the scalar is exactly 1, and conclude that
$$ \frac{1}{|G|} \sum_{g \in G} tr(gA) tr(gB) = A B^t $$
