# Statistical Model and its Topology

Consider the following setup:

Let $$(\Omega, \mathcal{B}, \mu)$$ be a probability space and let us denote $$\mathcal{M}_{\mu}$$ the set of all densities of all probability measures equivalent to $$\mu$$: $$\mathcal{M}_{\mu} = \{ f \in L^1(\mu) : f > 0 \ \mu\text{-a.e. and} \ \mathbb{E}(f) = 1 \}.$$ The probability measure whose density is $$f$$ will be denoted by $$f \cdot \mu$$ and its expectation by $$\mathbb{E}_f(\cdot)$$, where $$\mathbb{E}(\cdot)$$ denotes the expectation with respect to the reference measure $$\mu$$.

The paper that I am reading Paper is written that

"The more obvious geometry of $$\mathcal{M}_{\mu}$$ comes from the fact that it is a convex subset of $$L^1(\mu)$$. One of the key ideas in the geometric theory of statistical models is that embedding in $$L^1(\mu)$$ is not natural for statistical purposes."

$$\textbf{My question is}$$: Why is embedding $$\mathcal{M}_{\mu}$$ in $$L^1(\mu)$$ considered not natural for statistical purposes? What are the alternatives and the reasoning behind these alternatives in the context of the geometric theory of statistical models?

• It'd probably be useful if you drop a reference to the paper in the question, just to give proper context. Also worth explicitly delineating the quote (maybe use a quote block, or at least " ") just so its clear what the paper says and what is you editorialising. Commented Jul 9 at 17:59
• The paper attributes the claim to this book which I was unable to find online. Commented Jul 10 at 9:26
• The alternative constructions are the ones discussed in the paper you link; Pistone, Sempi 95. We can consider instead various topologies on the ensemble of probability distributions. The typical case is the weak-topology but they discuss other ones with more differential-geometry theory behind them. For statistical practice, I'm not quite sure if I see detailed consequences from such abstract considerations? I'll read their paper and return to this question afterwards. Commented Jul 10 at 9:32
• I find the sentence to be quite vague, but very high level, I suspect that the property they're talking about is the fact that the L_1 metric (which is the same as total variation) does not tensorise well, i.e., if $f^{\otimes n}$ is the distribution of $n$ iid samples from a density $f$, then it is difficult to relate $\| f^{\otimes n} - g^{\otimes n}\|_1$ to $\|f-g\|_1$ in a manner that effectively captures how the increase in $n$ makes distinguishing distributions (and thus doing statistics) easier. Commented Jul 10 at 16:27
• Typical examples of such alternative structures are captured by expressing differences in distributions via an $f$-divergence (e.g., KL divergence is standard in information geometry), or via Hellinger-type metrics (which take $f \in L_1$ and map it to $f^{\alpha} \in L_{1/\alpha},$ and then study the problem in $L_{1/\alpha}$. Hellinger corresponds to $\alpha = 1/2$). The basic property that makes these useful is that the difference b/w $f^{\otimes n}, g^{\otimes n}$ becomes a simple function of the difference b/w $f,g$ and $n$ (e.g., $KL(f^{\otimes n}\|g^{\otimes n}) = n KL(f\|g))$. Commented Jul 10 at 16:29