# Why does $E_g\left[-\frac{(x-\mu)^2}{2\sigma^2}\right] = -\sigma^2_0+\frac{(\mu-\mu_0)^2}{\sigma^2}$

I'm reading about information criteria and I bumped into an example, where the author tries to approximate a true data generating normal function $g(x|\mu_0, \sigma^2_ 0)$ with an approximation model:

$$f(x|\mu,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right).$$

The goal is to have $f(x|\mu,\sigma^2)\approx g(x|\mu_0, \sigma^2_ 0)$. Anyways, in the text I'm reading the author then writes the following:

$$\log f(x|\mu,\sigma^2) = -\frac{1}{2}\log2\pi\sigma^2-\frac{(x-\mu)^2}{2\sigma^2},$$

which is completely clear. But this next one is where I start to get trouble, he writes:

$$E_g[\log f(x|\mu,\sigma^2)] = -\frac{1}{2}\log2\pi\sigma^2-\sigma^2_0+\frac{(\mu-\mu_0)^2}{\sigma^2},$$

where expectation has been calculated with respect to the true distribution $g$. It therefore seems to me that:

$$E_g\left[-\frac{(x-\mu)^2}{2\sigma^2}\right] = -\sigma^2_0+\frac{(\mu-\mu_0)^2}{\sigma^2}.$$

I don't immediately see why this is true, so I tried to show it myself:

$$E_g\left[-\frac{(x-\mu)^2}{2\sigma^2}\right]=-\frac{1}{2\sigma^2}\left(E_g\left[x^2-2x\mu+\mu^2\right]\right) = -\frac{1}{2\sigma^2}\left(E_g\left[x^2\right]-2\mu E_g\left[x\right]+E_g\left[\mu^2\right]\right)$$

$$=-\frac{1}{2\sigma^2}\left(E_g\left[x^2\right]-2\mu\mu_0+\mu^2\right).$$

In the above I have used the assumption that $E_g\left[x\right]=\mu_0$ and I also assume that $E_g\left[(x-\mu_0)^2\right]=\sigma^2_0$. So now I need to show that:

$$-\frac{1}{2\sigma^2}\left(E_g\left[x^2\right]-2\mu\mu_0+\mu^2\right)=-\sigma^2_0+\frac{(\mu-\mu_0)^2}{\sigma^2}$$

and now I start to doubt my derivation.

Question: Am I going in the right direction? Is there some mistake in my reasoning here that I have made?

UPDATE: to see my reference book where my question originates, please see page 62 in the book Information Criteria and Statistical Modeling

• Your calculations seems correct. Can you share the source? – user144410 Feb 23 '18 at 9:45
• Equation 3.7 on page 32 is correct and agrees with the computations. Where does the author gives your highlighted equation above? – user144410 Feb 23 '18 at 9:56
• @user144410 thank you for your comment, I have gotten little bit further in my derivation but I can't yet get the result the author gets. Yes I can, the book I'm reading this from is: Information Criteria and Statistical Modeling by Konishi et al. springer.com/la/book/9780387718866 page 62. – jjepsuomi Feb 23 '18 at 9:58
• @user144410 The highlighted equation is not anywhere in the book, this is my own deduction. Please check the two equations after Equation 3.109 at page 62. The two equations are what gave rise to my post and questions. – jjepsuomi Feb 23 '18 at 9:59

$$E_g[x^2] = \sigma_0^2 + \mu_0^2$$ Therefore, \begin{aligned} E_g[x^2] - 2 \mu \mu_0 + \mu^2 &= \sigma_0 + \mu_0^2 - 2 \mu \mu_0 + \mu^2\\ &=\sigma_0^2 + (\mu - \mu_0)^2 \end{aligned} and it holds that $$E_g\left[-\frac{(x-\mu)^2}{2\sigma^2}\right] = - \frac{\sigma^2_0+(\mu-\mu_0)^2}{2\sigma^2}.$$
This agrees with equation (3.7) in the book you are refering to. However, when there is a relation between $\sigma_0$ and $\sigma$ the formula can be reduced.
It seems in the book you are referring to that the specified model has a $\sigma$ given by the empirical variance. Try writing this in terms of the true variance.
• Hi @user144410 and thank you very much :) Okay, so you're saying that, there is probably a relation between $\sigma^2$ and $\sigma^2_0$ which results in the simplified version (in page 62) of the Equation 3.7 in page 32? – jjepsuomi Feb 23 '18 at 10:10