Evaluating integrals in the paper Auto-Encoding Variational Bayes This is the first time that I'm asking on this site so apologies in advance if it's not quite the usual standard. 
I'm going thorough the paper Auto-Encoding Variational Bayes https://arxiv.org/abs/1312.6114 and in page 10 appendix B there is a simple enough looking equation that I don't fully understand how they got the results they have.
$$\int q_{\phi}(\mathbf{z})log\,p(\mathbf{z})d\mathbf{z} = ... $$
when I expand $log\,p(\mathbf{z})$ I get 
$$log\,p(\mathbf{z}) = log\, \mathcal{N}(\mathbf{z}; \mathbf{0}, \mathbf{I}) = 
 log\, det(2\pi I)^{-1/2}\,e^{-1/2 \,\mathbf{z}'I\mathbf{z}} = \frac{-J}{2}\, log \,2\pi \, -\frac{1}{2}\left\lVert \mathbf{z} \right\rVert ^{2}$$
where $J$ is the dimension of $\mathbf{z}$. From there it is easy to see that the term $\int q_{\phi}(\mathbf{z})\frac{-J}{2}\, log \,2\pi\, d\mathbf{z}$ evaluates to $\frac{-J}{2}\, log \,2\pi$ given that the integral over a density function evaluates to 1 which is in agreement with the paper. However I'm not sure how to proceed with the second part $\int q_{\phi}(\mathbf{z})\frac{-J}{2}\, \left\lVert \mathbf{z} \right\rVert ^{2} d\mathbf{z}$.
I'll really appreciate if anyone that knows could please explain what is it that I'm misunderstanding.
Thanks.
 A: It's actually not terribly difficult, the key observation is that the integral actually splits into a bunch of single variable integrals,
\begin{align}
\int q_\phi({\bf z})\|{\bf z}\|_2^2d{\bf z}&=\int q_\phi({\bf z})(z_1^2+\ldots+z_n^2)d{\bf z}\notag\\
&=\int q_\phi({\bf z})z_1^2d{\bf z} + \ldots + \int q_\phi({\bf z})z_n^2d{\bf z}\notag\\
&=\int q_\phi(z_1)z_1^2dz_1 + \ldots + \int q_\phi(z_n)z_n^2dz_n\notag.
\end{align}
Now since $q_\phi(z_i) = N(\mu, \sigma^2)$, the above expression can be rewritten in expected value form as,
$$\sum_{i=1}^n\text{E}Z_i^2,$$
and using the identity that $\text{Var}X = \text{E}X^2 - (\text{E}X)^2$, we obtain,
$$\sum_{i=1}^n\text{E}Z_i^2=\sum_{i=1}^n(\text{E}Z_i)^2 + \text{Var}Z_i=\sum_{i=1}^n\mu_i^2 + \sigma_i^2.$$
A: Perhaps a bit late, but may be useful for someone needing the full derivation of KL divergence:
Worked derivation of KL-divergence
By definition of KL divergence:
$$
\begin {equation}
D_{KL}(Q||P) = \int Q(X) \log \frac{Q(X)}{P(X)}dX
\end {equation}
$$
Or, put another way:
$$
\begin {equation}
\int Q(X) \bigg[\log P(X) - \log Q(X) \bigg]dX = \int Q(X)\log P(X)dX - \int Q(X) \log Q(X)dX
\end {equation}
$$
First, let's expand $\log Q(X)$:
$$ \log Q(X) = \log \bigg[\frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2 \sigma^2}}\bigg] $$
Then using the rule $\log(XY) = \log(X) + \log(Y)$
$$
\log Q(X) = \log \frac{1}{\sqrt{2\pi\sigma^2}} - \frac{(x-\mu)^2}{2 \sigma^2}
$$
So we can compute the 1st term in eq. (2) $\int Q(X)\log Q(X)$:
$$
\int Q(X)\log Q(X) = \int Q(X) \bigg[\log\frac{1}{\sqrt{2\pi\sigma^2}} - \frac{(x-\mu)^2}{2 \sigma^2} \bigg] dX
$$
Expand the brackets:
$$
\int Q(X)\log Q(X) = \int Q(X) \log\frac{1}{\sqrt{2\pi\sigma^2}} dX - \int Q(X) \frac{(x-\mu)^2}{2 \sigma^2} dX
$$
In the first term we take out of the integral the $\log\frac{1}{\sqrt{2\pi\sigma^2}}$ because it doesn't depend on X, what's left is equal to 1 (by the definition of normal distribution). Because $\log\frac{1}{\sqrt{2\pi\sigma^2}} = \log(1)-\log \sqrt{2\pi} + \log\sqrt{\sigma^2} = -\frac{1}{2}\log 2\pi - \frac{1}{2}\log \sigma^2$, and by definition $Var(X) = \int Q(X) (x-\mu)^2 dX = \sigma^2 $, we can write:
$$
-\frac{1}{2}\log 2\pi -\frac{1}{2}\log\sigma^2 - \frac{1}{2 \sigma^2}\sigma^2 = 
-\frac{1}{2}\log 2\pi -\frac{1}{2} + \log\sigma^2
$$
Because we have are going to have a batches of $\mu$ and $\sigma$, we need to sum over the batch length $J$:
$$\boxed{-\frac{1}{2}\log 2\pi -\frac{1}{2}\sum_{j=1}^{J} \bigg[1 + \log\sigma_j^2 \bigg]}$$
Now let's do the second term in eq. (2) $\int Q(X)\log P(X)dX$
First, let's take the log apart:
$$ \log P(X) = \log \bigg[\frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2 \sigma^2}}\bigg] $$
Since $P(X) \stackrel{}{\sim} \mathcal{N} (0, I) $ is a normal distribution with $\mu = 0$ and $\sigma = 1$, we can write:
$$ \log P(X) = \log \bigg[\frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}\bigg] = 
-\frac{1}{2}\log 2\pi -\frac{x^2}{2}||x||^2
$$
$$\int Q(X) \bigg[ -\frac{1}{2}\log 2\pi -\frac{x^2}{2}||x||^2 \bigg]dX$$
$$-\frac{1}{2} \log 2\pi \int Q(X) dX - \frac{1}{2}\int Q(X) ||x||^2 dX
$$
Since by definition (see Wikipedia) $Var(X) = E[X^2] - E[X]^2 = E[X^2] - \mu^2$ and $\int Q(X) ||x||^2 dX$ is the expectation of  $||X||^2$, i.e. $\int Q(X) ||x||^2 = E[X^2] = Var(X) + E[X]^2 = \sigma^2 + \mu^2 $, we can write:
$$
-\frac{1}{2}\log 2\pi - \frac{1}{2} E[X^2] = -\frac{1}{2}\log 2\pi - \frac{1}{2} E[\mu^2+\sigma^2]
$$
Since $\mu$ and $\sigma$ are vectors, we need to replace the expectation with a sum:
$$
-\frac{1}{2}\log 2\pi - \frac{1}{2} E[X^2] =$$
$$
\boxed {-\frac{1}{2}\log 2\pi - \frac{1}{2} \sum_{j=1}^{J}[\mu_j^2+\sigma_j^2]}
$$
Finally we can put the 1st and 2nd terms together to get $D_{KL}$
$$
D_{KL} = -\frac{1}{2}\log 2\pi -\frac{1}{2}\sum_{j=1}^{J} \big(1 + \log\sigma_j^2 \big) - \Bigg(-\frac{1}{2}\log 2\pi - \frac{1}{2} \sum_{j=1}^{J}\big(\mu_j^2+\sigma_j^2\big) \Bigg)
$$
$$
\boxed {D_{KL} = -\frac{1}{2}
\sum_{j=1}^{J} \big(1 + \log\sigma_j^2 - \mu_j^2 - \sigma_j^2 \big)}
$$
