In my backstory, I was introduced to the geometric concept of infinity rather young, through reading about the inversive plane. In the course of learning calculus, I'm pretty sure I formed a concept of $\pm \infty$ lying at the endpoints of the real line (although they weren't real numbers themselves), and understood limits in terms of that.
By the time I was introduced to the extended real numbers, it was simply putting a name to the prior concept, and providing a framework to work with them in a clear and precise fashion. (and similarly for the projective real numbers)
Fast forward 20 years later, and through interactions with people here at MSE, I find there is a lot of antagonism towards the concept of the extended real numbers. I don't mean things like "it would be confusing to teach them in introductory calculus" -- I mean things like "the extended reals $\pm \infty$ are best thought of in terms of limits rather than as actual points" or even "thou shalt not develop a concept of $x$ approaching something as $x \to +\infty$" as well as some patently false claims (e.g. "$+\infty$ cannot be a mathematical object; it can merely be a a 'concept'").
I had previously brushed off those opinions, but they seem pervasive enough that I felt I should ask the titular question: is there any good reason for this antagonism? Or is there any good reason to avoid understanding calculus in terms of the extended reals (when they are suitable objects to do so)?