I am learning representation theory from Serre's book by myself. Currently I am reading about induced representations, but I don't understand the importance. The concept looks strange and the definition appears quite complicated compared to the topics discussed before it. Can someone briefly exposit its importance?
You may think of it like this, maybe it helps a bit. If you've got a representation for a larger group$~G$, you can always restrict to a subgroup$~H$ to get a representation of the latter. In particular you can restrict all irreducibeles for$~G$ to$~H$, and see what they do there; some may remain irreducible, some may decompose after restriction. But if you already know things about the subgroup$~H$ (for instance you may have classified its irreducibles), and which to find out things about$~G$, all this does not help much. You need a way to go in the other direction, take a representation of$~H$ and build one of$~G$ out of it. Since $G$ is more complicated than$~H$, it is too naive to assume that you can always extend a given $H$-representation to a $G$-representation by just defining the action of the new elements (in other words view your old representation as the restriction of a newly defined one); you may need to increase the dimension in order to extend the action. Induction does precisely this in a way that only uses the old representation and the group structure of $H$ within $G$ as ingredients; no choices are required. It is not the inverse operation of restriction, but it they are "adjoint" in a sense made precise by the Frobenius reciprocity (for instance the irreducibles $\rho$ of $G$ that show up when decomposing and induced irreducible $H$-representation $\sigma$ are precisely those for which $\sigma$ shows up in the decomposition of the restriction to$~H$ of$~\rho$; in fact the multiplicities of occurrence will be identical).
To see the use of induced representations, you might want to study easy examples, such as the (complex) representations of the dihedral group, using the easily described structure of the representations of its subgroup of rotations (as the subgroup is commutative, its irreducible representations are all of dimension$~1$). In general induction from relatively "easy" (but not too small) subgroups is often an important tool in studying representations of a given group.
An analogy is change of base fields for vector spaces. Every complex vector space can be seen as a real vector space by restriction (forget the multiplication by$~\mathbf i$), but not every real vector space can be so obtained (the dimension has to be even). However every real vector space can be complexified to a complex vector space, making its complex dimension equal to the original real dimension (and the real dimension has doubled). This is what implicitly goes on behind the scenes when they tell you to interpret a real square matrix as a complex one for the purpose of finding eigenvalues (the corresponding eigenspaces have no direct meaning for the original real space, since they live in its complexification). In this analogy, induction of representations corresponds to complexification.
If you want to compute the character table of a group one of the ways you search for new irreducibles is to just come up with some random representation and then use the orthogonality relations to remove the irreducible summands that you are already aware of. If what's left over is irreducible then you've found a new one. Of course, it wont always be the case that the representation you picked had exactly one irreducible summand that was unknown to you, so it's helpful to have many ways of creating representations that way the odds are higher that that will happen with something you can create.
Inducing is one of the standard go-to's for creating new representations.
There are of course many other reasons, this is only one example, but calculating character tables is a pretty standard thing to teach in a first course about representations so hopefully this example will be relevant to you.