Consider the following setup:

Let $(\Omega, \mathcal{B}, \mu)$ be a probability space and let us denote $\mathcal{M}_{\mu}$ the set of all densities of all probability measures equivalent to $\mu$: $$ \mathcal{M}_{\mu} = \{ f \in L^1(\mu) : f > 0 \ \mu\text{-a.e. and} \ \mathbb{E}(f) = 1 \}. $$ The probability measure whose density is $f$ will be denoted by $f \cdot \mu$ and its expectation by $\mathbb{E}_f(\cdot)$, where $\mathbb{E}(\cdot)$ denotes the expectation with respect to the reference measure $\mu$.

The paper that I am reading Paper is written that

"The more obvious geometry of $\mathcal{M}_{\mu}$ comes from the fact that it is a convex subset of $L^1(\mu)$. One of the key ideas in the geometric theory of statistical models is that embedding in $L^1(\mu)$ is not natural for statistical purposes."

$\textbf{My question is}$: Why is embedding $\mathcal{M}_{\mu}$ in $L^1(\mu)$ considered not natural for statistical purposes? What are the alternatives and the reasoning behind these alternatives in the context of the geometric theory of statistical models?

  • 2
    $\begingroup$ It'd probably be useful if you drop a reference to the paper in the question, just to give proper context. Also worth explicitly delineating the quote (maybe use a quote block, or at least " ") just so its clear what the paper says and what is you editorialising. $\endgroup$ Commented Jul 9 at 17:59
  • $\begingroup$ The paper attributes the claim to this book which I was unable to find online. $\endgroup$ Commented Jul 10 at 9:26
  • $\begingroup$ The alternative constructions are the ones discussed in the paper you link; Pistone, Sempi 95. We can consider instead various topologies on the ensemble of probability distributions. The typical case is the weak-topology but they discuss other ones with more differential-geometry theory behind them. For statistical practice, I'm not quite sure if I see detailed consequences from such abstract considerations? I'll read their paper and return to this question afterwards. $\endgroup$ Commented Jul 10 at 9:32
  • $\begingroup$ I find the sentence to be quite vague, but very high level, I suspect that the property they're talking about is the fact that the L_1 metric (which is the same as total variation) does not tensorise well, i.e., if $f^{\otimes n}$ is the distribution of $n$ iid samples from a density $f$, then it is difficult to relate $\| f^{\otimes n} - g^{\otimes n}\|_1$ to $\|f-g\|_1$ in a manner that effectively captures how the increase in $n$ makes distinguishing distributions (and thus doing statistics) easier. $\endgroup$ Commented Jul 10 at 16:27
  • $\begingroup$ Typical examples of such alternative structures are captured by expressing differences in distributions via an $f$-divergence (e.g., KL divergence is standard in information geometry), or via Hellinger-type metrics (which take $f \in L_1$ and map it to $f^{\alpha} \in L_{1/\alpha},$ and then study the problem in $L_{1/\alpha}$. Hellinger corresponds to $\alpha = 1/2$). The basic property that makes these useful is that the difference b/w $f^{\otimes n}, g^{\otimes n}$ becomes a simple function of the difference b/w $f,g$ and $n$ (e.g., $KL(f^{\otimes n}\|g^{\otimes n}) = n KL(f\|g))$. $\endgroup$ Commented Jul 10 at 16:29

1 Answer 1


A tangential answer, but I want to share a great paper on this topic: What is a statistical model? , McCullaugh 2002

The author provides a detailed characterization of statistical models in terms of category theory. It's a tough read, but it does highlight (in very unnatural and heavy, but accurate(!), language) some key properties that a statistical model should have. In particular, the author discusses how the intuitive properties: "my model should give predictions" "the predictions should be sensible" "the model should aim to be universal" can be encoded into category-theory language over models.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .