$\newcommand{\arcsec}{\operatorname{arcsec}}$This expression can be integrated without trig substitution, but I wanted to try using it anyways and got a different answer.
Starting with finding the indefinite integral $$\int x^3\sqrt{x^2-9} \, dx$$ I then substituted $x = 3\sec\theta$, $\theta = \arcsec(\frac{x}{3})$, and $d\theta = \frac{3}{\sqrt{x^2-9} |x|} \, dx$. Plugging these values into the original integral, $$\int 27\sec^3\theta\sqrt{9\sec^2\theta - 9}\frac{3}{\sqrt{9\sec^2\theta-9} |3\sec\theta|} \, d\theta$$ $$\int \frac{81\sec^3\theta3\tan\theta}{3\tan\theta |3\sec\theta|} \, d\theta$$ $$\int \frac{81\sec^3\theta}{|3\sec\theta|} \, d\theta$$ $$\int 27\sec^2\theta \, d\theta$$ $$27\tan\theta + C$$ Now substituting $x$ back in and simplifying: $$27\tan(\arcsec(\frac{x}{3})) + C$$ $$9\operatorname{sgn}(x)\sqrt{x^2-9} + C$$
This does not seem close at all to the solution I found by $u$-substitution, $$\frac{(x^2-9)^\frac{3}{2}(x^2+6)}{5} + C$$
I am relatively new to integration so I think I made a mistake. What did I do wrong? Any help appreciated.