Generally what I think makes such series so good is that the format forces the authors to explain non-trivial and often non-elementary mathematics in accessible and inspiring way. The concentration of mathematically "clever" and "cool" both fascinates and challenges. They also often expose parts and perspectives of mathematics that are largely missing in standard texts and approaches. Of course, not every book in a series is equally good, so I will list some that I find particularly outstanding. But I haven't read all of them, so it doesn't mean that the rest are sub par, and my assessment of level only applies on average.
Not a series per se but similar in spirit and close to the upper undergraduate level of AMS Student Mathematical Library (and cheap) are some (most are just texts) of the Dover Books on Mathematics: Riemann's Zeta Function; Three Pearls of Number Theory; Geometry and Light; Counterexamples in Topology; Regular polytopes; Beauty of Geometry; Asymptotic methods in Analysis; Satan, Cantor and Infinity; Hyperbolic Functions.
Some of Cambridge University Press' London Mathematical Society Student Texts are more than typical texts, and they are at the right level too: Prime Number Theorem, Undergraduate Algebraic Geometry, Elliptic Functions, Young Tableaux. Also good but very short are their Outlooks, and Canadian Mathematical Society's Treatises in Mathematics series.
MAA and Cambridge University Press support Dolciani Mathematical Expositions, which is freshman/sophomore level: Charming Proofs; Diophantus and Diophantine equations; Logic as Algebra. MAA's Classroom Resource Materials also has some entries at this level: Paradoxes and Sophisms in Calculus; Counterexamples in Calculus; Explorations in Complex Analysis; Which Numbers are Real?; Real Infinite Series.
The ones below, especially Mir's, are generally less advanced, high school/freshman level. Still, I grew up reading such booklets, and learned from them more than from most formal studying, they also guided my interests later and helped select topics which I wanted to pursue in depth.
AMS's Mathematical World: A Mathematical Gift, I, II, III; Mathematical Ciphers: From Caesar to RSA; Kvant Selecta (collections of best articles from Russian math journal for advanced high school kids); Stories about Maxima and Minima.
MAA's New Mathematical Library: Game Theory and Strategy; Geometry of Numbers; Numbers: rational and irrational; Ingenuity in Mathematics; Geometric transformations; Uses of Infinity.
Mir's Little Mathematics Library: Proof in Geometry; Solving Equations in Integers; Inequalities; Areas and Logarithms; Remarkable Curves.