# General Convergence of Exponential of Function to Dirac Delta

Let $$f:\mathcal{X}\rightarrow \mathcal{F}$$ be a function with a unique, positive maximum at $$x^*=\arg\sup_{x}f(x)$$ where $$\mathcal{X}$$ and $$\mathcal{F}$$ are both bounded. Let $$f$$ be locally smooth about $$x^*$$, that is $$\exists\ \Delta>0\ s.t. f(x)\in\mathbb{C}^2\ \forall\ x\in\{x\vert \lVert x-x^*\rVert<\Delta$$ }. Is it possible to prove that the distribution $$p_\epsilon(x)=z_\epsilon\exp\left(\frac{f(x)}{\epsilon}\right)$$, where $$z_\epsilon=\left(\int\exp\left(\frac{f(x)}{\epsilon}\right)dx\right)^{-1}$$ is the normalisation constant, converges to a Dirac delta distribution in the limit $$\epsilon\rightarrow 0$$? I've been trying to formally show that

$$a)\quad\lim_{\epsilon\rightarrow0}\int z_\epsilon\exp\left(\frac{f(x)}{\epsilon}\right)\varphi(x)dx=\varphi(x^*)$$

for any continuous $$\varphi(x)$$, but I have limited background in analysis. Moreover, if this result holds, can it be extended to any joint $$p_\epsilon(x)=z_\epsilon\exp\left(\frac{f(x)}{\epsilon}\right)q(x)$$ with $$z_\epsilon=\left(\int\exp\left(\frac{f(x)}{\epsilon}\right)q(x)dx\right)^{-1}$$ where $$q(x)$$ is another distribution, that is can we prove:

$$b)\quad\lim_{\epsilon\rightarrow0}\int z_\epsilon\exp\left(\frac{f(x)}{\epsilon}\right)q(x)\varphi(x)dx=\varphi(x^*),$$

again for any $$\varphi(x)$$ is continuous?

For those interested, the context of this comes from reinforcement learning where we seek a policy $$\pi(a\vert s)$$ that is greedy with respect to a value function $$Q(a,s)$$, that is $$\pi(a\vert s)=\delta(a\in\arg\max_{a'}Q(a',s))$$.

• That $f$ is bounded and continuous is not enough to make $\int \exp\left(\frac{f(x)}{\epsilon}\right) \, dx$ finite, so $p_\epsilon$ might not be defined. – md2perpe Apr 30 at 9:26
• Ah, yeah forgot to say the domain of $f(x)$ is bounded too. Have edited. – Mattiegf Apr 30 at 9:48
• I also think that $p_\epsilon \to \delta$ if $f$ has a maximum in one point. But I have not been able to show it yet. – md2perpe Apr 30 at 14:12
• I've edited the question to make the notion of local smoothness more precise. – Mattiegf May 16 at 0:53

\begin{align} g(x):= f(x)-f(x^*). \end{align}
Note, $$g(x)\le 0$$ with equality at $$g(x^*)=0$$. Substituting $$f(x)=g(x)+f(x^*)$$ into $$p_\varepsilon(x)$$: \begin{align} p_\varepsilon(x)=&\frac{\exp\left(\frac{g(x)+f(x^*)}{\varepsilon}\right)}{\int_\mathcal{X}\exp\left(\frac{g(x)+f(x^*)}{\varepsilon}\right)dx},\\ =&\frac{\exp\left(\frac{g(x)}{\varepsilon}\right)\exp\left(\frac{f(x^*)}{\varepsilon}\right)}{\int_\mathcal{X}\exp\left(\frac{g(x)}{\varepsilon}\right)\exp\left(\frac{f(x^*)}{\varepsilon}\right)dx},\\ =&\frac{\exp\left(\frac{g(x)}{\varepsilon}\right)}{\int_\mathcal{X}\exp\left(\frac{g(x)}{\varepsilon}\right)dx}.\quad (1) \end{align} Now, substituting (1) into the limit in a) yields: \begin{align} \lim_{\varepsilon\rightarrow0}\int_{\mathcal{X}} \varphi(x)p_\varepsilon(x)dx=\lim_{\varepsilon\rightarrow0}\left(\int_{\mathcal{X}} \varphi(x)\frac{\exp\left(\frac{g(x)}{\varepsilon}\right)}{\int_\mathcal{X}\exp\left(\frac{g(x)}{\varepsilon}\right)dx}dx\right).\label{eq:limit_g} \end{align} Using the substitution $$u:=\frac{(x^*-x)}{\sqrt{\varepsilon}}$$ to transform the integrals in (1), we obtain \begin{align} \lim_{\varepsilon\rightarrow0}\int_{\mathcal{X}} \varphi(x)p_\varepsilon(x)dx&=\lim_{\varepsilon\rightarrow0}\left(\int_{\mathcal{U}} \varphi(x^*-\sqrt{\varepsilon}u)\frac{\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)}{\int_\mathcal{U}\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\sqrt{\varepsilon}du}\sqrt{\varepsilon}du\right),\\ &=\lim_{\varepsilon\rightarrow0}\left(\frac{\int_{\mathcal{U}} \varphi(x^*-\sqrt{\varepsilon}u)\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)du}{\int_\mathcal{U}\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)du}\right).\quad (2) \end{align} We now find $$\lim_{\varepsilon\rightarrow0}\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)$$: Using $$\mathrm{L'H\hat{o}pital's}$$ rule to the second derivative with respect to $$\sqrt{\epsilon}$$, we have: \begin{align} \lim_{\varepsilon\rightarrow0}\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)&=\lim_{\varepsilon\rightarrow0}\left(\frac{\partial_{\sqrt{\varepsilon}}g(x^*-\sqrt{\varepsilon}u)}{\partial_{\sqrt{\varepsilon}}\varepsilon}\right),\\ &=\lim_{\varepsilon\rightarrow0}\left(\frac{\partial_{\sqrt{\varepsilon}}f(x^*-\sqrt{\varepsilon}u)}{\partial_{\sqrt{\varepsilon}}\varepsilon}\right),\\ &=\lim_{\varepsilon\rightarrow0}\left(\frac{-u^\top\nabla f(x^*-\sqrt{\varepsilon}u)}{2\sqrt{\varepsilon}}\right),\\ &=\lim_{\varepsilon\rightarrow0}\left(\frac{\partial_{\sqrt{\varepsilon}}\left(u^\top\nabla f(x^*-\sqrt{\varepsilon}u)\right)}{\partial_{\sqrt{\varepsilon}}(2\sqrt{\varepsilon})}\right),\\ &=\lim_{\varepsilon\rightarrow0}\left(\frac{u^\top\nabla^2 f(x^*-\sqrt{\varepsilon}u)u}{2}\right),\\ &=\frac{u^\top\nabla^2 f(x^*)u}{2}. \end{align} The integrand in the numerator of (2) therefore converges pointwise to $$\varphi(x^*)\exp\left(\frac{u^\top\nabla^2 f(x^*)u}{2}\right)$$, that is \begin{align} \lim_{\varepsilon\rightarrow0}\left(\varphi(x^*-\sqrt{\varepsilon}u)\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\right)=\varphi(x^*)\exp\left(\frac{u^\top\nabla^2 f(x^*)u}{2}\right),\quad (3) \end{align} and the integrand in the denominator converges pointwise to $$\exp\left(\frac{u^\top\nabla^2 f(x^*)u}{2}\right)$$, that is \begin{align} \lim_{\varepsilon\rightarrow0}\left(\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\right)=\exp\left(\frac{u^\top\nabla^2 f(x^*)u}{2}\right).\quad (4) \end{align} From the second order sufficient conditions for $$f(x^*)$$ to be a maximum, we have $$u^\top\nabla^2 f(x^*)u \le 0$$ $$\forall\ u\in\mathcal{U}$$ with equality only when $$u=0$$. This implies that (3) and (4) are both bounded functions.
By definition, we have $$g(x^*-\sqrt{\epsilon} u)\le0\ \forall\ u\in\mathcal{U}$$, which implies that $$|\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)|\le1$$. Consequently, the integrand in the numerator of (2) is dominated by $$\lVert \varphi(\cdot)\rVert_\infty$$, that is \begin{align} \left\lvert\varphi(x^*-\sqrt{\varepsilon}u)\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\right\rvert\le\lVert \varphi(\cdot)\rVert_\infty,\quad (5) \end{align} and the integrand in the denominator is dominated by $$1$$, that is \begin{align} \left\lvert\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\right\rvert\le1.\quad (6) \end{align}
\begin{align} \lim_{\varepsilon\rightarrow0}\int_{\mathcal{X}} \varphi(x)p_\varepsilon(x)dx&=\lim_{\varepsilon\rightarrow0}\left(\frac{\int_{\mathcal{U}} \varphi(x^*-\sqrt{\varepsilon}u)\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)du}{\int_\mathcal{U}\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)du}\right),\\ &=\frac{\int_{\mathcal{U}}\lim_{\varepsilon\rightarrow0}\left( \varphi(x^*-\sqrt{\varepsilon}u)\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\right)du}{\int_\mathcal{U}\lim_{\varepsilon\rightarrow0}\left(\exp\left(\frac{g(x^*-\sqrt{\varepsilon}u)}{\varepsilon}\right)\right)du},\\ &=\frac{\int_{\mathcal{U}} \varphi(x^*)\exp\left(u^\top\nabla^2 f(x^*)u\right)du}{\int_\mathcal{U}\exp\left(u^\top\nabla^2 f(x^*)u\right)du},\\ &=\varphi(x^*)\frac{\int_{\mathcal{U}} \exp\left(u^\top\nabla^2 f(x^*)u\right)du}{\int_\mathcal{U}\exp\left(u^\top\nabla^2 f(x^*)u\right)du},\\ &=\varphi(x^*). \end{align}