Importance of Representation Theory Representation theory is a subject I want to like (it can be fun finding the representations of a group), but it's hard for me to see it as a subject that arises naturally or why it is important. I can think of two mathematical reasons for studying it:


*

*The character table of a group is packs a lot of information about the group and is concise.

*It is practically/computationally nice to have explicit matrices that model a group.
But there must certainly be deeper things that I am missing. I can understand why one would want to study group actions (the axioms for a group beg you to think of elements as operators), but why look at group actions on vector spaces? Is it because linear algebra is so easy/well-known (when compared to just modules, say)?
I am also told that representation theory is important in quantum mechanics. For example, physics should be $\mathrm{SO}(3)$ invariant and when we represent this on a Hilbert space of wave-functions, we are led to information about angular momentum. But this seems to only trivially invoke representation theory since we already start with a subgroup of $\mathrm{GL}(n)$ and then extend it to act on wave functions by $\psi(x,t) \mapsto \psi(Ax,t)$ for $A$ in $\mathrm{SO}(n)$.
This Wikipedia article on particle physics and representation theory claims that if our physical system has $G$ as a symmetry group, then there is a correspondence between particles and representations of $G$. I'm not sure if I understand this correspondence since it seems to be saying that if we act an element of G on a state that corresponds to some particle, then this new state also corresponds to the same particle. So a particle is an orbit of the $G$ action? Anyone know of good sources that talk about this?
 A: Groups very, very rarely appear abstractly in (mathematical) nature: when we encounter them, it is almost always in the form of a representation (a linear representation, a permutation representation, a non-linear representation in the form of automorphisms of an algebra, of a variety, of a manifold, of a group (!)).
It is thus only natural that the study of representation theory be important! I would go further, turn your question backwards, and claim that group theory is important because it allows us to study the representations of groups.
A: *

*The representation theory of finite groups can be used to prove results about finite groups themselves that are otherwise much harder to prove by "elementary" means. For instance, the proof of Burnside's theorem (that a group of order $p^a q^b$ is solvable).  A lot of the classification proof of finite simple groups relies on representation theory (or so I'm told, I haven't read the proof...).

*Mathematical physics. Lie algebras and Lie groups definitely come up here, but I'm not familiar enough to explain anything. In addition, the classification of complex simple Lie algebras relies on the root space decomposition, which is a significant (and nontrivial) fact about the representation theory of semisimple Lie algebras.

*Number theory.  The nonabelian version of L-functions (Artin L-functions) rely on the representations of the Galois group (in the abelian case, these just correspond to sums of 1-dimensional characters).  For instance, the proof that Artin L-functions are meromorphic in the whole plane relies on (I think) Artin Brauer's theorem (i.e., a corollary of the usual statement) that any irreducible character is an rational integer combination of induced characters from cyclic subgroups -- this is in Serre's Linear Representations of Finite Groups. Also, the Langlands program studies  representations of groups $GL_n(\mathbb{A}_K)$ for $\mathbb{A}_K$ the adele ring of a global field. This is a generalization of standard "abelian" class field theory (when $n=1$ and one is determining the character group of the ideles).

*Combinatorics. The representation theory of the symmetric group has a lot of connections to combinatorics, because you can parametrize the irreducibles explicitly (via Young diagrams), and this leads to the problem of determining how these Young diagrams interact. For instance, what does the tensor product of two Young diagrams look like when decomposed as a sum of Young diagrams? What is the dimension of the irreducible representation associated to a Young diagram? These problems have a combinatorial flavor.
I should add the disclaimer that I have not formally studied representation theory, and these are likely to be an unrepresentative sample of topics (some of which I have only vaguely heard about).
A: Although many years later, for the sake of completeness -regarding the last question on references about uses in physics- i believe one might find lots of interest on the classic text by J. Baez and J. Huerta: 
The Algebra of Grand Unified Theories
Lots of the mathematical physics developed during the last decades of the 20th century is reviewed there, in a particularly didactic way. 
A: Particles correspond to specific vectors in a representation, not to $G$-orbits! The reason has to do with "symmetry breaking."  The $8$ particles in the meson octet correspond to a basis of a certain $8$-dimensional representation of the group $\mathrm{SU}(3)$ called the "adjoint representation." At high enough energies these particles would be indistinguishable. But at low energies the "$\mathrm{SU}(3)$ symmetry has been broken" and the particles become distinguishable.
Another good physics example that's easier to understand is that the orbital states of electrons in atoms correspond to representations of the group $\mathrm{SO}(3)$ of symmetries of space (well, really $\mathrm{SU}(2)$ if you want to incorporate spin). Try reading a standard quantum mechanics textbook for a little bit of this picture and then try thinking about it in terms of representation theory.
A: Representation theory plays a big role in the group-theoretic approach to special functions.
For example, Willard Miller showed that the powerful Infeld-Hull factorization / ladder method - widely exploited by physicists - is equivalent to the representation theory of four local Lie groups. This lie-theoretic approach served to powerfully unify and "explain" all prior similar attempts to provide a unfied theory of such classes of special functions, e.g. Truesdell's influential book An Essay Toward a Unified Theory of Special Functions. Below is the first paragraph of the introduction to Willard Miller's classic monograph Lie theory and special functions

This monograph is the result of an attempt to understand the role
played by special function theory in the formalism of mathematical
physics. It demonstrates explicitly that special functions which arise in
the study of mathematical models of physical phenomena and the
identities which these functions obey are in many cases dictated by
symmetry groups admitted by the models. In particular it will be shown
that the factorization method, a powerful tool for computing eigenvalues
and recurrence relations for solutions of second order ordinary differential
equations (Infeld and Hull), is equivalent to the representation
theory of four local Lie groups. A detailed study of these four groups and
their Lie algebras leads to a unified treatment of a significant proportion
of special function theory, especially that part of the theory which is
most useful in mathematical physics.

See also Miller's sequel Symmetry and Separation of Variables. Again I quote from the preface:

This book is concerned with the relationship between symmetries of a
linear second-order partial differential equation of mathematical physics,
the coordinate systems in which the equation admits solutions via separation
of variables, and the properties of the special functions that arise in
this manner. It is an introduction intended for anyone with experience in
partial differential equations, special functions, or Lie group theory, such
as group theorists, applied mathematicians, theoretical physicists and
chemists, and electrical engineers. We will exhibit some modern group-theoretic
twists in the ancient method of separation of variables that can be
used to provide a foundation for much of special function theory. In
particular, we will show explicitly that all special functions that arise
via separation of variables in the equations of mathematical physics can
be studied using group theory. These include the functions of Lame, Ince,
Mathieu, and others, as well as those of hypergeometric type.
This is a very critical time in the history of group-theoretic methods in
special function theory. The basic relations between Lie groups, special
functions, and the method of separation of variables have recently been
clarified. One can now construct a group-theoretic machine that, when
applied to a given differential equation of mathematical physics, describes
in a rational manner the possible coordinate systems in which the equation
admits solutions via separation of variables and the various expansion
theorems relating the separable (special function) solutions in distinct
coordinate systems. Indeed for the most important linear equations, the
separated solutions are characterized as common eigenfunctions of sets of
second-order commuting elements in the universal enveloping algebra of
the Lie symmetry algebra corresponding to the equation. The problem of
expanding one set of separable solutions in terms of another reduces to a
problem in the representation theory of the Lie symmetry algebra.

See Koornwinder's review of this book for a very nice concise introduction to the group-theoretic approach to separation of variables.
A: The idea of considering the representations of a group $G$ as a group of linear operators to ultimately gain informations on the group $G$ itself is an important istance of the general process of "linearization" that is employed systematically in many areas of mathematics.
In down-to-earth (and possibly vague) terms, I would say that the only general class of problems that we are able to resolve fully are those which are linear, meaning that involve only vector spaces--best if finitely dimensional :-) --and linear maps between them. When we deal with objects that are intrinsically non-linear, a widespread strategy is to try to attach to them some linear "gadgets" that in some way preserve, in a different form, some of the structure of the original object.
Two basic examples that should be well-known to every math student:


*

*The construction of the tensor product reduces multilinear algebra to linear algebra.

*The construction of the homotopy and (co)homology groups of a topological space is an attempt to "understand topology via groups and linear spaces".
The linear representations of a group $G$ respond to the same principle: they are a "linear" set of data attached to $G$ which hopefully can serve to characterize $G$.  
A: One comment about your sentence "this seems to only trivially invoke representation theory". It might be surprising, but such obvious representations are actually the source of interesting mathematics, and a lot of effort of representation theorists is devoted to studying them.
More precisely: start with a group (in your example $SO(n)$) acting on a space $X$ (in your example $\mathbb R^n$), and look at the space of functions on $X$ (let me write it $\mathcal F(X)$; in a careful treatment, one would have to think about whether we wanted continuous, smooth, $L^2$, or some other kind of functions, but I will suppress that kind of technical consideration).
Then, as you observe, there is a natural representation of $G$ on $\mathcal F(X)$.
You are right that from a certain point of view this seems trivial, because the representation is obvious.  Unlike when one first learns rep'n theory of finite groups, where one devotes a lot of effort to constructing reps., in this context, the rep. stares you in the face.
So how can this be interesting?
Well, the representation $\mathcal F(X)$ will almost never be irreducible. How does this representation decompose?
Suddenly we are looking at a hard representation theoretic problem.  


*

*First, we have to work out the list of irreps of $G$ (which is much
like what one does in a first course on rep'n theory of finite groups).

*Second, we have to figure out how $\mathcal F(X)$ decomposes, which involves representation theory (among other things, you have to develop methods for investigating this sort of question), and also often a lot of analysis (because typically $\mathcal F(X)$ will be infinite dimensional, and may be a Hilbert space, or have some other similar sort of topological vector space structure which should be incorporated into the picture). 
I don't think I should say too much more here, but I will just give some illustrative examples:


*

*If $ X = G = S^1$ (the circle group, say thought of as $\mathbb R/\mathbb Z$)
acting on itself by addition, then the solution to the problem of decomposing
$\mathcal F(S^1)$ is the theory of Fourier series.  (Note that a function on $S^1$ is the
same as a periodic function on $\mathbb R$.)

*If $ X = G = \mathbb R$, with $G$ acting on itself by addition, then
the solution to the above question (how does $\mathcal F(\mathbb R)$ decompose under the
action of $\mathbb R$) is the theory of the Fourier transform.

*If $ X = S^2$ and $G = SO(3)$ acting on $X$ via rotations, then decomposing $\mathcal F(S^2)$ into irreducible representations gives the theory of spherical harmonics.
(This is an important example in quantum mechanics; it comes up for example in the theory of the hydrogen atom, when one has a spherical symmetry because the electron orbits the nucleus, which one thinks of as the centre of the sphere.)

*If $ X = SL_2(\mathbb R)/SL_2(\mathbb Z)$ (this is the quotient of a Lie group by a discrete subgroup, so is naturally a manifold, in this case of dimension 3),
with $G = SL_2(\mathbb R)$ acting by left multiplication, then the problem of decomposing $\mathcal F(X)$ leads to the theory of modular forms and Maass forms, and is the first example in the more general theory of automorphic forms.
Added: Looking over the other answers, I see that this is an elaboration on  AD.'s answer.
A: Representation theory is one of the keys to the Fourier transform on groups (and co-set spaces etc.). For this matter, I recommend this book:
Unitary representations and harmonic analysis: an introduction
 By Mitsuo Sugiura
If you take a look at the Wikipedia page about representation theory, you will find several applications.
A: Certainly, group actions used to model dynamical systems, for instance, Markov Chains, find extremely handy the results from representation theory.
For example, suppose you have a deck of cards, and every time you choose two cards at random and transpose them. How long you have to wait for the deck to be randomized? The (exact) answer to this question lies exclusively in the area of representation theory! See for example the book Harmonic Analysis on Finite Groups by Ceccherini-Silberstein, Scarabotti, and Tolli.
