# What is known about pairs of "physical" SDEs and "statistical" SDEs?

Background: Recall that a Langevin motion on a Riemannian manifold $$(M, g)$$ in $$\mathbb{R}^D$$ can be written down as the solution to the SDE in a local chart $$U\subset \mathbb{R}^d$$ (open) $$dY_t = [\nabla_g \cdot g^{-1}(Y_t) -\nabla_g V(Y_t)]dt+\sqrt{2g^{-1}(Y_t)}dB_t.$$ Here $$V:U\to \mathbb{R}$$ is the potential function, a scalar, $$g$$ is the metric tensor, $$\nabla_g f$$ for scalar $$f$$ is the manifold gradient, and $$\nabla_g \cdot\vec{f}$$ for vector fields $$\vec{f}$$ is the manifold-divergence. Precisely, $$\nabla_g f := g^{-1} \nabla f$$ and $$\nabla_g \cdot \vec{f} = (\det g)^{-1/2} \nabla \cdot (\sqrt{\det g} \vec{f})$$. We extend the manifold divergence to matrix fields by applying it row-wise.

It is well known that $$Y$$ has steady-state density $$p_\infty(y) \propto e^{-V(y)}\sqrt{\det g(y)},$$ i.e. its steady-state measure has Radon-Nikodym derivative $$e^{-U(y)}$$ with respect to the volume measure $$\nu(dy) = \sqrt{\det g(y)}dy$$ on $$M$$.

Now consider the scenario where either the potential or the geometry, or both, introduces some parameters $$\theta$$ (could be a vector or scalar). In that case, we might clarify our notation so that $$p_\infty(y;\theta) \propto e^{-V(y;\theta)} \sqrt{\det g(y;\theta)}$$ For each $$\theta$$ we must assume $$V(\cdot ;\theta)$$ is a potential, and $$g(\cdot;\theta)$$ is a metric tensor, etc. Now, we have a parametric distribution. Provided this varies smoothly over $$\theta$$, we can compute the Fisher information metric

$$I(\theta)_{ij} =-\mathbb{E}_{Y_\infty}(\partial_{\theta_i \theta_j} \log p(Y_\infty;\theta))$$ Let $$\Theta$$ be the parameter space. Now, we potentially have another Riemannian manifold, a statistical manifold $$(\Theta, I)$$, and we can play the same sort of game and write down a Langevin motion in the parameter space. An interesting potential is to use the negative log likelihood associated to $$p_\infty$$, i.e. $$\tilde{V}(\theta) := -\log L(y_1,\dotsc, y_n;\theta),$$ where $$L(y_1,\dotsc, y_n;\theta) = \prod_{i=1}^n p_\infty(y_i;\theta)=e^{-\sum_{i=1}^n V(y_i)} \prod_{i=1}^n \sqrt{\det g(y_i)},$$ where $$Y_i=y_i$$ are the observed variates of IID $$Y_i\sim Y_\infty$$. Let $$I_n(\theta)=nI(\theta)$$. Then the Langevin motion in the statistical space given by $$d\theta_t = [\nabla_{I_n} I_n^{-1}(\theta_t)-\nabla_{I_n} \tilde{V}(\theta_t)]dt+\sqrt{2I_n^{-1}(\theta_t)}dW_t$$ has steady-state $$f_\infty(\theta) \propto e^{-\tilde{V}(\theta)} \sqrt{\det I_n(\theta)}$$ $$=L(x_1,\dotsc, x_n;\theta) J(\theta)$$ where $$J(\theta) = \sqrt{\det I_n(\theta)}$$ is Jefferey's prior. But we recognize this from Bayesian statistics! This is just saying that $$f_\infty(\theta)=f_\infty(\theta|x_1,\dotsc, x_n)$$ is the posterior distribution of $$\theta$$ given $$X_1,\dotsc, X_n$$ using Jefferey's prior.

Question: I am interested in what is known about the pair $$Z_t=(\theta_t, Y_t)$$, i.e. solutions to the system

$$d\theta_t = [\nabla_{I_n} I_n^{-1}(\theta_t)-\nabla_{I_n} \tilde{V}(\theta_t)]dt+\sqrt{2I_n^{-1}(\theta_t)}dW_t$$

$$dY_t = [\nabla_g \cdot g^{-1}(Y_t;\theta) -\nabla_g V(Y_t;\theta)]dt+\sqrt{2g^{-1}(Y_t;\theta)}dB_t,$$ where $$B$$ and $$W$$ are independent Brownian motions. In particular, what is the steady state distribution of $$Z_t$$? Is there anything interesting we can say about the relationship between the "physical" motion $$Y_t$$ on $$(M,g)$$ and the "statistical" motion of $$\theta_t$$ on $$(\Theta, I_n)$$? Note if $$\theta$$ is a parameter of $$g$$ and not just the potential, then the above SDE system has a motion $$Y$$ on a random manifold!

(I apologize if this is too open ended and if anyone has suggestions to narrow it, I will be open to them.)

Case when only the potential is parametric: If only the potential $$V$$ depends on a parameter, then we can easily derive two facts. Let $$Z(\theta)$$ be the normalization "constant" for $$p_\infty$$, or as they say in statistical mechanics, the partition function: $$Z(\theta) = \int_U e^{-V(y;\theta)} \sqrt{\det g(y)} dy$$. Then,

1. $$-\frac{\partial \log Z}{\partial \theta_i} = \mathbb{E}\left(\frac{\partial V}{\partial \theta_i}(Y_\infty;\theta)\right)$$
2. $$I(\theta)_{ij} = \operatorname{Cov}\left(\frac{\partial V}{\partial \theta_i}(Y_\infty;\theta), \frac{\partial V}{\partial \theta_j}(Y_\infty;\theta)\right)$$ which matches our intuition that the Fisher information should only depend on the potential and not the metric tensor $$g$$, in this case.

It is hard to resist hoping that this special case might be useful or have another interesting things to say about it...

A concrete example: To give some intuition here is a simple example. Let $$Y$$ be a Langevin motoion with harmonic potential on Euclidean space with a diffusion scale $$1/\beta$$: $$dY_t = -k Y_t dt + \sqrt{2/\beta_t}dB_t$$ and $$\beta_t$$ solves the SDE $$d\beta_t = [2n^{-1}\beta_t+\beta_t(1-\beta_t k \overline{y^2})]dt+2n^{-1/2}\beta_t dW_t$$ where $$\overline{y^2}:=\frac{1}{n}\sum_{i=1}^n y_i^2$$ where $$Y_i=y_i$$ and $$Y_i\sim Y_\infty$$ IID. Note that given $$\beta_t=\beta$$ $$Y_\infty \sim \mathcal{N}(0, (k\beta)^{-1})$$ and computing the Fisher information, log-likelihood of this and all of the above necessary calculus will yield the SDE for $$\beta_t$$. Here is a plot the pair $$(y_t, \beta_t)$$ of the system when $$\tilde{V}=0$$ (first row) and when $$\tilde{V}$$ is the negative-log likelihood (second row).

The plots show the steady-state is still approached even for the Brownian motion on the statistical space ($$\tilde{V}=0$$).

Summary Question: Is there any known literature studying pairs of "physical" SDEs and "statistical" SDEs like this?