First of all expansion around singular points are important/interesting for applications. For example, Bessel equation
often arises in problems with axial symmetry. Its singular point $x=0$ corresponds precisely to approaching the symmetry axis.
Second, expanding around an arbitrarily chosen point, in general, does not allow to obtain all expansion coefficients in a nice closed form - one only finds a recursion formula they satisfy. Whereas the recursion relation for the corresponding expansion coefficients around a singularity is often solvable. Example: the very same Bessel equation has a nice series solution
A more philosophical viewpoint: any 2nd order differential equation can be written as a rank 2 1st order linear system. Suppose we have its $2\times 2$ fundamental matrix solution $\Phi(x)$. Being analytically continued along a closed path on the Riemann sphere, $\Phi$ will be multiplied by a constant matrix, called monodromy matrix, which depends only on the homotopy class of the path.
If we choose $\Phi$ arbitrarily, the monodromy matrix for the loop encircling a particular singular point is nothing special. However, when $\Phi$ is constructed from the solutions obtained by the Frobenius method, the monodromy matrix is special (e.g. diagonal in the Bessel case). This is to say that such solutions are rather distinguished from the complex-analytic point of view. Yet in other words, Frobenius method suggests a particular "good" basis in the solution space. For instance, this choice of the basis is largely responsible for the existence of differentiation and recursion formulas for Bessel functions.