3
$\begingroup$

I'm trying to gain some intuition on the Boltzmann distribution (also called Gibbs canonical distribution). The definition is very broad: it's an exponential probability distribution that is a function of an energy and temperature, $$ P(x) \propto e^{-\epsilon(x)/kT(x)} $$ where $\epsilon$ and $T$ are the energy and temperature of the state $x$, and $k$ is the Boltzmann constant.

An example is the 1D Gaussian distribution, which has pdf $$P(x) \propto e^{-\frac{(x-\mu)^2}{2\sigma^2}}.$$ Apparently this is the Boltzmann distribution corresponding to the potential energy $$\epsilon(x) = \frac{(x-\mu)^2}{2\sigma^2}.$$

How do I interpret this? I can see that if a point $x$ is far from the mean, then it has high potential energy.

In statistics, the idea of "entropy" is related to the spread of a distribution. Is this saying that a high entropy distribution has high potential energy?

$\endgroup$
7
  • $\begingroup$ So is $kT(x) = C$ a constant in this regard? $\endgroup$
    – Gregory
    Commented Mar 17, 2021 at 20:19
  • $\begingroup$ I ask because intuitively to me, high temperature means particles will be moving faster. This should lead to higher variability in their location. So I would expect $T(x)$ to be related to $\sigma$. And since this is a question about context/interpretation, it seems important to distinguish. $\endgroup$
    – Gregory
    Commented Mar 17, 2021 at 20:21
  • $\begingroup$ @Gregory That would make sense, I think -- the spread of a distribution is related to its entropy, and entropy is related to temperature... these are the vague connections in my mind $\endgroup$
    – 900edges
    Commented Mar 17, 2021 at 20:30
  • $\begingroup$ I'm not an expert, but still... First, I don't see how a Boltzmann distribution can be a generalization of a Gaussian, seeing that the first one is a discrete distribution (with infinite support) while the other is a continuous one. Second, my admittedly limited experience in biochemistry have made me familiar with the Boltzmann factor, but I have never seen the $\frac{(x-\mu)^2}{2\sigma^2}$ form anywhere. Isn't that the parametrization of some families of distributions? It might be that Boltzmann can be expressed in those terms, but I've never seen it used in (my limited) practice. $\endgroup$ Commented Mar 17, 2021 at 21:10
  • 2
    $\begingroup$ @Gregory given my lack of expertise, I'll promptly return to lurking, but I thought important to raise the point about the fundamental nature of the distributions. Incidentally, I checked and could see that you're absolutely right, this form is quite common. Glad I learned something today :) $\endgroup$ Commented Mar 17, 2021 at 21:28

2 Answers 2

1
$\begingroup$

The potential energy ${1\over2}k(x-\mu)^2$ is exactly the well known potential energy of a harmonic spring with offset $\mu$. A spring in a thermal bath where the kinetic energy can be neglected, or can be integrated over will have a position distribution proportional to $e^{-{1\over 2}\beta k (x-\mu)^2}$ where $\beta = {1\over \kappa T}$

$\endgroup$
4
  • $\begingroup$ I see, so in context does that say that the distribution of the spring follows a Gaussian distribution? (This makes sense to me in the 1D case) $\endgroup$
    – 900edges
    Commented Mar 18, 2021 at 15:42
  • $\begingroup$ The "thermalized" spring is distributed as a Gaussian. You can see this by solving the Orenstein Ulenbeck equation $\endgroup$
    – user619894
    Commented Mar 18, 2021 at 15:59
  • $\begingroup$ Good to know! If you don't mind, I'll edit your answer a bit to fill in the details for my own and others' reference $\endgroup$
    – 900edges
    Commented Mar 18, 2021 at 16:11
  • $\begingroup$ @900edges somehow I erased your edit by accident, but I tried to put the relevant info in the answer. $\endgroup$
    – user619894
    Commented Mar 18, 2021 at 16:49
1
$\begingroup$

Well, one could write pages here. This is a short glimpse from my perspective.
Let $V$ be a nice (convex, smooth, etc.) potential. We know, that the gradient flow \begin{equation} \dot{x}=-\nabla V(x) \end{equation} will relax to the equilibrium, attained at the minimum of $V$. Moreover, $V$ decreases along solutions.
The overdamped Langevin equation is a stochastic pertubation of the gradient flow dynamic above: \begin{equation} dX_{t}=-\nabla V(X_{t})dt+\sqrt{2T}dB_{t} \end{equation} which naturally arises e.g. in statistical mechanics. Classical diffusion theory now tells us that this SDE has an invariant measure of Gibbs-type \begin{equation} \nu_{T}(dx)=\frac{1}{Z_{T}}e^{-\frac{V(x)}{T}}dx \end{equation} where $Z_{T}$ is a normalising constant, often referred to as partition function in statistical mechanics. The Gaussian setting corresponds to a quadratic potential and to an Ornstein-Uhlenbeck process for the SDE. The effect of the random pertubation decreases with a decrease in temperature. On the level of the invariant measure this amounts to a large deviation result for the family $\lbrace\nu_{T}\rbrace_{T>0}$. In particular $\nu_{T}$ weakly converges to the point measure concentrated at the minimiser of $V$. This observation is used in various optimisation algorithms, e.g. simulated annealing.

And since you mentioned entropy: the laws $\mu_{t}$ corresponding to $X_{t}$ evolve according to the Fokker-Planck equation \begin{equation} \partial_{t}\mu=T\Delta\mu-\nabla\cdot(\mu\nabla V). \end{equation} It's an easy exercise to check that the relative entropy $H(\cdot||\mu_{T})$ decreases along solution, i.e. is a Lyapunov function. In fact, since the pioneering work of Felix Otto, we know that this Fokker-Planck equation can itself be interpreted as a gradient flow in an infinite dimensional setting, namely \begin{equation} \dot{\mu}=-\operatorname{grad}_{W}H(\mu||\nu_{T}) \end{equation} where $\operatorname{grad}_{W}$ denotes the formal gradient within Ottos Wasserstein geometry. Since the relative entropy is convex and minimal if $\nu=\mu_{T}$, we recover again the relaxation of the laws towards the invariant Gibbs measure.

$\endgroup$
5
  • $\begingroup$ A lot of these words are over my head but is general interpretation correct?: the boltzmann/gibbs distribution gives the (invariant) distribution of a particle following langevin dynamics on the potential surface, and the temperature tells us the concentration of the measure? $\endgroup$
    – 900edges
    Commented Mar 18, 2021 at 16:00
  • $\begingroup$ I'm not sure why large $T$ should give greater concentration in the potential minimizer... shouldn't it this occur for small $T$ since this means the particle doesn't have enough energy to escape? $\endgroup$
    – 900edges
    Commented Mar 18, 2021 at 16:01
  • $\begingroup$ Sure, $\nu_{T}\to\delta_{x^{*}}$ as $T\to 0$. $\endgroup$
    – Tobsn
    Commented Mar 18, 2021 at 18:54
  • $\begingroup$ What is $T$ in your answer? Is $x$ defined on $[0, T]$? $\endgroup$ Commented May 14, 2022 at 12:05
  • $\begingroup$ If you read the answer and the original question carefully, there can be no confusion on what $T$ is. $\endgroup$
    – Tobsn
    Commented May 14, 2022 at 12:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .