Can every Riemannian manifold be written as a statistical manifold? Given a sufficiently nice PDF $f:M\times \Theta \to \mathbb{R}$ where $M\subset \mathbb{R}^D$ and $\Theta\subset \mathbb{R}^p$ if the Fisher Information matrix
$$I_{ij}(\vec{\theta})=\mathbb{E}\left[\left(\partial_{\theta_i} \log f(\vec{X}, \vec{\theta})\right) \cdot \left(\partial_{\theta_j} \log f(\vec{X}, \vec{\theta})\right)\bigg|\theta\right]$$
is positive definite (PD) then it forms a metric tensor in the variable $\theta$ and we can say that $(\Theta, I)$ is a manifold, called a statistical manifold. Thus, for nice enough PDFs, we can always find an associated a statistical manifold.
Question:
Given a Riemannian manifold $(\Theta,g)$ with metric tensor $g$, can we find a PDF $f:M\times \Theta\to\mathbb{R}$ (for some $M$) such that $I(\vec{\theta})=g(\vec{\theta})$? If so, is such a PDF unique? In other words: is every metric tensor the Fisher information metric for some RV $X$? Another way to ask is whether every metric tensor $g$ on $\Theta$ can be written as
$$g(\theta) = - \mathbb{E}\left[\nabla_\theta^2 \log f(\vec{X}, \vec{\theta})\bigg| \theta\right],$$
for some sufficiently nice $f$?
Some thoughts:
I would wager that the class of manifolds is larger than the class of statistical manifolds (those whose metric tensors are the FIM of some RV). But this is just a rough guess and I am not sure what tools to use to approach this question. If anybody has suggestions or hints I would gladly expend more effort on this and add some attempts.
 A: Statistical manifolds were introduced by Lauritzen in this work (even if they were already known by people working in information geometry). The idea behind is precisely the one mentioned by @Didier in their comment. Note, however, that Lauritzen uses a symmetric, 3-covariant tensor called the skewness tensor $T$ instead of the dually related connections mentioned in the comment.
Then, Hong Van Le proved here that every compact $C^{1}$-statistical manifold $(M,g,T)$ is actually a statistical model (see also here and here). This means that you can represent points in $M$ as probability distributions on a given outcome space $\mathcal{X}$ in such a way that $g$ coincides with the Fisher-Rao metric tensor and $T$ with the so-called Amari-Cencov tensor coming from realizing $M$ inside $\mathcal{P}(\mathcal{X})$ (probability distributions on $\mathcal{X}$).
Unfortunately, I do not remember the details of the proof, nor I remember if meaningful comments are made concerning the non-compact case.
