Why is infinitary logic regarded as mysterious? I am looking for references to the "mysteriousness" of infinitary logic, preferably from people in the field, and even more preferably, with some kind of explanation as to why. 
For example, is it that the semantics is underdetermined? Is it that the metaphysics is hard to conceive? 
A reference without explanation would also be fine. 
 A: Infinitary formulas are inherently more complicated than finitary ones in various fundamental senses.


*

*Formulas in usual finitary first-order logic (fine,in a countable language) can be represented by natural numbers in a reasonable way; this is just Godel numbering. Note that this is more than just being countable, I'm saying that the set of finitary formulas is "usefully countable." By contrast, the set of $\mathcal{L}_{\omega_1\omega}$-formulas - this is the simplest infinitary logic - is uncountable. Each $\mathcal{L}_{\omega_1\omega}$-formula can be reasonably represented by a set of natural numbers. But for a variety of reasons, $\mathcal{P}(\mathbb{N})$ is viewed as a much more mysterious object than $\mathbb{N}$. For example, there are $\mathcal{L}_{\omega_1\omega}$-formulas which can't be "computably described!"

*Moving beyond the complexity of the logic as a set, the deductive procedure is also much weirder: compactness fails, and proofs are infinitary objects. Basically, infinitary logics don't have nearly as well-behaved a model theory or proof theory as classical first-order logic; ignoring philosophy, they're simply more mathematically complicated and harder to prove things about (e.g. in lieu of compactness we have the much more technical Barwise compactness). Also, as per the above paragraph, the complexity of (say) the set of infinitary validities is measured by descriptive set theory instead of, say, computability theory.

*Finally, when we pass to more complicated infinitary logics, things get even worse. Obviously, a $\mathcal{L}_{\omega_2\omega}$-formula is even harder to "describe" than a $\mathcal{L}_{\omega_1\omega}$-formula. But it's even worse than when we allow infinitary quantifiers. $\mathcal{L}_{\omega_1\omega_1}$ is the logic we get when we allow countable Boolean combinations and quantification over $\omega$-tuples (instead of just finitary quantification), and this is truly dreadful: e.g. we can (in an appropriate sense) express "the set of constructible reals is countable" as an $\mathcal{L}_{\omega_1\omega_1}$-sentence, but this sentence's truth value is set-theoretically dependent in a very strong sense. So everything's bonkers.
Basically, infinitary logic pushes us into set theory to a degree which finitary logic doesn't. (Although we shouldn't take the absoluteness of finitary first-order logic too blithely.)
