This looks like a Bessel differential equation with an extra $x^4,x^6$ term but I am not able to figure out to proceed in solving it. Any idea how to proceed to solve this equation? Any help/reference will be highly appreciated.
$$x^2\frac{d^2y}{dx^2}+x\frac{dy}{dx} + (\alpha x^2-\beta x^4 - \gamma x^6)y = 0$$ $\alpha,\beta,\gamma$ are all constant.
There's a general form of Bessel differential equation on the Mathworld website but I am unable to connect it with the equation here.
Maple doesn't find a closed-form solution in general. In the case $\gamma = 0$ it does find the solution using Kummer functions:
$$ y \! \left(x \right) = c_{1} {\mathrm e}^{-\frac{\sqrt{\beta}\, x^{2}}{2}} M\! \left(-\frac{-2 \sqrt{\beta}+\alpha}{4 \sqrt{\beta}}, 1, \sqrt{\beta}\, x^{2}\right)+c_{2} {\mathrm e}^{-\frac{\sqrt{\beta}\, x^{2}}{2}} U\! \left(-\frac{-2 \sqrt{\beta}+\alpha}{4 \sqrt{\beta}}, 1, \sqrt{\beta}\, x^{2}\right) $$
In the general case, you can use the Frobenius method to find series solutions. The indicial equation is $r^2 = 0$. Thus there should be a series solution of the form $\sum_{n=0}^\infty a_n x^n$ and a second solution involving a logarithmic term. The first few terms are
$$ y \! \left(x \right) = c_{1} \left(1-\frac{1}{4} \alpha \,x^{2}+\left(\frac{\alpha^{2}}{64}+\frac{\beta}{16}\right) x^{4}+\left(-\frac{1}{2304} \alpha^{3}-\frac{5}{576} \beta \alpha +\frac{1}{36} \gamma \right) x^{6}+\ldots\right)+c_{2} \left(\ln \! \left(x \right) \left(1-\frac{1}{4} \alpha \,x^{2}+\left(\frac{\alpha^{2}}{64}+\frac{\beta}{16}\right) x^{4}+\left(-\frac{1}{2304} \alpha^{3}-\frac{5}{576} \beta \alpha +\frac{1}{36} \gamma \right) x^{6}+\ldots\right)+\left(\frac{\alpha}{4} x^{2}+\left(-\frac{\beta}{32}-\frac{3 \alpha^{2}}{128}\right) x^{4}+\left(\frac{37}{3456} \beta \alpha -\frac{1}{108} \gamma +\frac{11}{13824} \alpha^{3}\right) x^{6}+\ldots\right)\right) $$
