Kullback-Leibler divergence of two exponential distributions with different scale parameters The question is as follows: 
"Calculate the Kullback-Leibler divergence between two exponential distributions with different scale parameters. When is it maximal?"
I have tried something but I come to a wrong conclusion (at least comparing with Wikipedia). 
Let the KL-divergence between the approximating distribution $p_\theta$ and the "true" distribution $p_{\theta_0}$ be defined as
$M(\theta) = P_{\theta_0} \log \frac{P_\theta}{P_{\theta_0}}$
The density of an exponential distribution is given by 
$p_\theta(x) = \theta e^{-\theta x}$
hence for the "true" distribution we have 
$p_{\theta_0}(x) = \theta_0 e^{-\theta_0 x}$
which gives
$P_{\theta_0}\log \frac{P_\theta}{P_{\theta_0}} = P_{\theta_0} \log \frac{\theta e^{-\theta x}}{\theta_0 e^{-\theta_0 x}}$
which we can simplify to 
$P_{\theta_0}\log \frac{P_\theta}{P_{\theta_0}} = P_{\theta_0} \log(\theta) - \log(\theta_0) - (\theta - \theta_0)x$
(This is where I think I might have made a possible mistake)
Because $x$ under the true distribution is exponentially distributed with scale parameter $\theta_0$ its mean is given by $1/\theta_0$ and as such we find that the KL-divergence is 
$M(\theta) = \log(\theta) - \log(\theta_0) - (\theta - \theta_0)\frac{1}{\theta_0} =\log(\theta) - \log(\theta_0) -\frac{\theta}{\theta_0}+1$
However, Wikipedia (link) gives the following KL-divergence for two exponential distributions
$M(\theta) =\log(\theta) - \log(\theta_0) +\frac{\theta}{\theta_0}-1$
My answer has the signs flipped, where did I make an error?
 A: Let $p_\theta$ be the true distribution and $p_\phi$ be the approximating one. Then the KL divergence is
\begin{align} 
\mathcal{D}_\text{KL}[p_\theta || p_\phi] 
&= \int_D p_\theta(x) \log\frac{p_\theta(x)}{p_\phi(x)} dx \\
&= \int_D \theta\exp(-\theta x)\log\frac{\theta\exp(-\theta x)}{\phi\exp(-\phi x)} dx \\
&= \int_D \theta\exp(-\theta x)\left[
\log(\theta)-\log(\phi)-\theta x + \phi x
\right] dx \\
&= \log\frac{\theta}{\phi}\underbrace{\int_D \theta\exp(-\theta x) dx}_{1 \text{ by}\int_D p_\theta=1}
+
(\phi-\theta)\underbrace{\int_D \theta\exp(-\theta x) x\, dx}_{\mathbb{E[x]}=\theta^{-1}\text{ if } x\,\sim\, p_\theta} \\
&= \log\theta - \log\phi +\frac{\phi - \theta}{\theta} \\
&= \log\theta - \log\phi +\frac{\phi}{\theta} - 1 \\
\end{align}
which is what Wikipedia gives.
As for "when is it maximal?" (kind of an odd question), we look at the derivative of the model parameter:
$$
\frac{\partial}{\partial \phi} \mathcal{D}_\text{KL}[p_\theta || p_\phi] 
=\frac{\partial}{\partial \phi} (\log\theta - \log\phi +\frac{\phi}{\theta} - 1)
= \frac{-1}{\phi} + \frac{1}{\theta}
$$
which shows unsurprisingly the distance is minimal when $\theta=\phi$. 
Note that the distance grows without bound as $\phi\rightarrow\infty$ or $\phi\rightarrow 0$ (noting that $\phi > 0$ by definition). Interestingly, $\phi\rightarrow \infty$ implies $\partial_\phi \mathcal{D}_\text{KL}[p_\theta || p_\phi] \rightarrow 1/\theta$ whereas $\partial_\phi \mathcal{D}_\text{KL}[p_\theta || p_\phi] \rightarrow -\infty$ as $\phi\rightarrow 0$.
Hopefully I didn't make a mistake!

Note: be careful because there are "variants" of the KL that are closely related and easy to mis-type, e.g.,
$$
-\mathcal{D}_\text{KL}[p_\theta || p_\phi] = \int p_\theta(x) \left(-\log\frac{p_\theta(x)}{p_\phi(x)}\right) = \int p_\theta(x) \log\frac{p_\phi(x)}{p_\theta(x)}
$$
$$
\mathcal{D}_\text{KL}[p_\phi|| p_\theta] = \int p_\phi(x) \log\frac{p_\phi(x)}{p_\theta(x)}
$$
