Strange statistical concepts like spaces of probability distributions, "metrics" like Fisher information or relative entropy, and convergence with respect to these quantities are necessitated in my research problems. I need to come up to speed on these tools quickly, but can't find any standard, rigorous treatment of them.
Can anyone share good references on this and the needed prerequisite knowledge?
It looks like to even start with this topic you need quite a lot of knowledge in stochastic processes (I am using the two volumes on diffusion by Rodgers as reference), functional analysis (using Folland and Conway), and maybe some differential geometry (using Lee's 3 books).
Edit: This conference paper linked by this question gives a strong argument that studying these things from a geometric perspective is misguided, which rules out information geometry references. But there is still plenty of motivation to study relative entropy and Fisher information.