Showing $E \{X_i X_j\} = \mu_i \mu_j + Q_{i,j}$ from characteristic function of Gaussian random variable. I'm confused as to how one differentiates (16.4) below so as to get
$E \{X_i X_j \} = \mu_i \mu_j + Q_{i,j}.$




 A: As you already know :
If $f : \mathbb{R} \to \mathbb{C}$ is a differentiable function then :
$$(\exp \circ f)'(x) = f'(x)\exp(f(x))$$
Let's denote $$f(u) = i \langle u, \mu \rangle - \frac 12 \langle u,Qu \rangle = i(u_1\mu_1 + ... + u_n \mu_n) - \frac{1}{2}\sum_{k,l}q_{k,l}u_ku_l$$
Then $$\partial_{u_i}f(u) = i\mu_i - \frac{1}{2}\sum_l q_{i,l}u_l - \frac{1}{2}\sum_k q_{k,i}u_k = i\mu_i - \sum_k q_{i,k}u_k$$  (since $Q$ is symetrical)
And $$\partial_{u_j}\partial_{u_i}f(u) = -q_{i,j}$$
In your case :
$$\partial_{u_i} \exp(f(u)) = (i\mu_i - \sum_k q_{i,k}u_k)\exp(f(u))$$
Differentiating a product, you get :
$$\partial_{u_j}\left((i\mu_i - \sum_k q_{i,k}u_k)\exp(f(u))\right) \\= -q_{i,j}\exp(f(u)) + \left(i\mu_i - \sum_k q_{i,k}u_k\right)\partial_{u_j}\exp(f(u))\\ =  -q_{i,j}\exp(f(u)) + \left(i\mu_i - \sum_k q_{i,k}u_k\right)\left(i\mu_j - \sum_k q_{j,k}j_k\right)\exp(f(u))$$
If you take $u = 0$ in both members you get the value $$-q_{i,j} + i^2 \mu_i \mu_j = i^2 E[X_i X_j]$$
A: $$\phi(s) =\int_{\mathbb{R}^n}f(x)e^{i\langle s,x\rangle}dx $$
Differentiating with respect to $s_i$ and then with respect to $s_j$
$$\frac{\partial^2 \phi}{\partial s_i \partial s_j} \bigg |_{s=0} = -E[X_i X_j]$$
In your case we get (using Einstein's summation convention)
$$\frac{\partial \phi}{\partial s_i} = \phi(s) \frac{\partial }{\partial s_i}\big(i s_k \mu _k - \frac{1}{2}Q_{kl}s_k s_l \big ) = \phi(s) \big[i \mu_k \delta_{ki}- (Q_{kl} \delta_{k i} s_l)\big  ]=\phi(s) \big[i \mu_i - Q_{il} s_l\big  ]$$
Differentiating again
$$\frac{\partial^2 \phi}{\partial s_i \partial s_j} = \frac{\partial \phi}{\partial s_j}\big[i \mu_i - Q_{il} s_l\big  ] - \phi Q_{ij}$$
And evaluating this at $s=0$
$$E[X_i X_j] =\mu_i\mu_j + Q_{ij}$$
