Boltzmann distribution as generalization of Gaussian I'm trying to gain some intuition on the Boltzmann distribution (also called Gibbs canonical distribution).  The definition is very broad: it's an exponential probability distribution that is a function of an energy and temperature, $$ P(x) \propto e^{-\epsilon(x)/kT(x)} $$ where $\epsilon$ and $T$ are the energy and temperature of the state $x$, and $k$ is the Boltzmann constant.
An example is the 1D Gaussian distribution, which has pdf $$P(x) \propto e^{-\frac{(x-\mu)^2}{2\sigma^2}}.$$ Apparently this is the Boltzmann distribution corresponding to the potential energy $$\epsilon(x) = \frac{(x-\mu)^2}{2\sigma^2}.$$
How do I interpret this? I can see that if a point $x$ is far from the mean, then it has high potential energy.
In statistics, the idea of "entropy" is related to the spread of a distribution. Is this saying that a high entropy distribution has high potential energy?
 A: The potential energy ${1\over2}k(x-\mu)^2$ is exactly the well known potential energy of a  harmonic spring with offset $\mu$. A spring in a thermal bath where the kinetic energy can be neglected, or can be integrated over will have a position distribution proportional to $e^{-{1\over 2}\beta k (x-\mu)^2}$  where $\beta = {1\over \kappa T}$
A: Well, one could write pages here. This is a short glimpse from my perspective.
Let $V$ be a nice (convex, smooth, etc.) potential. We know, that the gradient flow
\begin{equation}
\dot{x}=-\nabla V(x)
\end{equation}
will relax to the equilibrium, attained at the minimum of $V$. Moreover, $V$ decreases along solutions. 
The overdamped Langevin equation is a stochastic pertubation of the gradient flow dynamic above:
\begin{equation}
dX_{t}=-\nabla V(X_{t})dt+\sqrt{2T}dB_{t}
\end{equation}
which naturally arises e.g. in statistical mechanics. Classical diffusion theory now tells us that this SDE has an invariant measure of Gibbs-type
\begin{equation}
\nu_{T}(dx)=\frac{1}{Z_{T}}e^{-\frac{V(x)}{T}}dx
\end{equation}
where $Z_{T}$ is a normalising constant, often referred to as partition function in statistical mechanics. The Gaussian setting corresponds to a quadratic potential and to an Ornstein-Uhlenbeck process for the SDE. The effect of the random pertubation decreases with a decrease in temperature. On the level of the invariant measure this amounts to a large deviation result for the family $\lbrace\nu_{T}\rbrace_{T>0}$. In particular $\nu_{T}$ weakly converges to the point measure concentrated at the minimiser of $V$. This observation is used in various optimisation algorithms, e.g. simulated annealing.
And since you mentioned entropy: the laws $\mu_{t}$ corresponding to $X_{t}$ evolve according to the Fokker-Planck equation
\begin{equation}
\partial_{t}\mu=T\Delta\mu-\nabla\cdot(\mu\nabla V).
\end{equation}
It's an easy exercise to check that the relative entropy $H(\cdot||\mu_{T})$ decreases along solution, i.e. is a Lyapunov function. In fact, since the pioneering work of Felix Otto, we know that this Fokker-Planck equation can itself be interpreted as a gradient flow in an infinite dimensional setting, namely
\begin{equation}
\dot{\mu}=-\operatorname{grad}_{W}H(\mu||\nu_{T})
\end{equation}
where $\operatorname{grad}_{W}$ denotes the formal gradient within Ottos Wasserstein geometry. Since the relative entropy is convex and minimal if $\nu=\mu_{T}$, we recover again the relaxation of the laws towards the invariant Gibbs measure.
