Are there are any jobs in industry that involve more mathematics than just what a (good) undergrad program would cover: calculus, differential equations, real and complex analysis, some basic group theory, etc.? I'm stuck without an academic position but would love to do research, and I just don't see anything out there that isn't trivial. It's incredibly frustrating how little math there is available to me, and I have zero interest in doing mathematical modeling or becoming an actuary. So, what kinds of jobs--- salary, location, etc. are irrelevant--- involve math that's advanced enough to be interesting and challenging?
Depending on the areas of mathematics you find interesting, the answer may be an emphatic "Yes" or an emphatic "No".
Microsoft Research has a pretty strong group in Mathematics. Some members of that group, such as Michael Freedman, have background in very abstract mathematics and a world class research record, though their work at Microsoft is naturally geared more towards applications. The math involved in Microsoft Research's projects is certainly highly challenging, but whether it's interesting to you or not, only you can judge.
Google and Facebook also have strong research groups. I don't know what, exactly, you call "mathematical modeling", which is something you have zero interest in, but hopefully some of the focus areas of this group are of interest to you, and they certainly dedicate time for doing things that would qualify as research in academia.
The financial industry hires people with fairly deep mathematical background, usually deeper than what you'd get in a typical undergrad program. DE Shaw is known for having strong mathematical staff, and I have friends with math Ph.D.'s who were hired by Goldman Sachs and found both the interviews and the actual work very challenging. Renaissance Technologies is a pretty secretive firm but they are clearly hiring for mathematicians.
Another possible direction is bioinformatics. There's plenty of research being done in bioinformatics in industry, but again, I can't judge if this is an area you'd find appealing, or sufficiently "mathematical".
Are you interested in reservoir simulation?
Or finite element method?
Or aeronautical and aerospace engineering?