6
$\begingroup$

Let the following "gaussian-like" integral:

$$ I = \int_{\Re^n} \! \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp \left\{ -\frac{1}{2} (\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu}) \right\} \mathbf{x} \,\mathbf{d}\mathbf{x}, $$

where $\mathbf{x}=(x_1,\dots,x_n)^T$, $\mathbf{\mu} = (\mu_1,\dots,\mu_n)^T\in\Re^n$, and $\Sigma\in\mathbb{S}_{++}^{n}$.

Our main goal is to evaluate the above integral. To this end, let $\mathbf{x}-\mathbf{\mu}=S\mathbf{y}$, where $S$ is an $n \times n$ orthogonal matrix ($S^T=S^{-1}$) with determinant equal to $1$. Using this change of variable, the quadratic form shown in the integral written as:

$$ -\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})= -\frac{1}{2}\mathbf{y}^T(S^T\Sigma^{-1}S)\mathbf{y}= -\frac{1}{2}\mathbf{y}^T(S^{-1}\Sigma^{-1}S)\mathbf{y}= -\frac{1}{2}\mathbf{y}^TD\mathbf{y}, $$

where $D=\operatorname{diag}\{d_1,\dots,d_n\}$. As a result it is rewritten as follows:

$$ -\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})= -\frac{1}{2}\sum_{k=1}^{n} d_i y_i^2 $$

Moreover, $\mathbf{x}-\mathbf{\mu}=S\mathbf{y} \Rightarrow \mathbf{x}=S\mathbf{y}+\mathbf{\mu}=[\mathbf{s_1}\:\dots\:\mathbf{s_n}]\mathbf{y}+\mathbf{\mu}=(\mathbf{s_1}\cdot\mathbf{y}+\mu_1,\dots, \mathbf{s_n}\cdot\mathbf{y}+\mu_n)^T,$ where $\mathbf{s}_j$ is the $j$-th column of matrix $S$.

Using the above results, the original integral can be rewritten as follows:

$$ I = (I_1,\dots,I_n)^T, $$

where the $j$-th element of $I$ is given by:

$$ I_j = \int_{\Re^n} \! \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp \left\{ -\frac{1}{2}\sum_{k=1}^{n} d_i y_i^2 \right\} (\mathbf{s}_j\cdot\mathbf{y}+\mu_j) \,\mathbf{d}\mathbf{y}\\ = \int_{\Re^n} \! \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp \left\{ -\frac{1}{2}\sum_{k=1}^{n} d_i y_i^2 \right\} \mathbf{s}_j\cdot\mathbf{y} \,\mathbf{d}\mathbf{y}\\ + \int_{\Re^n} \! \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp \left\{ -\frac{1}{2}\sum_{k=1}^{n} d_i y_i^2 \right\} \mu_j \,\mathbf{d}\mathbf{y} \Rightarrow\\ I_j = \int_{\Re^n} \! \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp \left\{ -\frac{1}{2}\sum_{k=1}^{n} d_i y_i^2 \right\} \mathbf{s}_j\cdot\mathbf{y} \,\mathbf{d}\mathbf{y} + \mu_j $$

If we write the dot product $\mathbf{s}_j\cdot\mathbf{y}$ as

$$ \mathbf{s}_j\cdot\mathbf{y} = \sum_{r=1}^{n} s_{jr}y_r, $$

then the integral $I_j$ is given by:

$$ I_j = \int_{\Re^n} \! \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \left(\sum_{r=1}^{n} s_{jr}y_r\right) \exp \left\{ -\frac{1}{2}\sum_{k=1}^{n} d_i y_i^2 \right\} \,\mathbf{d}\mathbf{y} + \mu_j $$

I would like to ask, first, whether the whole approach above is correct or not(if so, please correct me), and, second, how could I evaluate the last integral, $I_j$. Does it converge, like the gaussian integral over $\Re^n$?

Thanks in advance! Every useful comment will be extremely appretiated!

$\endgroup$
7
  • $\begingroup$ Might you have intended $(2\pi)^{n/2}$ instead of $(2\pi)^{1/2}$? If so, then what you have if you omit the $\mathbf x$ just before the $d\mathbf x$ is a probability density (provided I'm right in understanding that you meant $\Sigma$ is a positive-definite symmetric matrix). Then the integral would just be the expected value, which is the vector $\mu$. $\endgroup$ Nov 15 '13 at 23:05
  • $\begingroup$ You don't seem to have expressed the determinant of the matrix as a function of $d_i$, $i=1,\dots,n$. $\endgroup$ Nov 15 '13 at 23:10
  • $\begingroup$ @lcv That's correct, I omitted that, but I think it's rather obvisous. By the way, I will add it to the original post. Thanks! $\endgroup$ Nov 15 '13 at 23:10
  • $\begingroup$ @MichaelHardy you're right. It's a typo, I will fix it. But, in that case, are you sure that $\mu$ is the right answer? What you state is rational, but can anyone else confirm? I'm confused! Thanks, anyway! $\endgroup$ Nov 15 '13 at 23:13
  • $\begingroup$ @MichaelHardy, I am not sure I understand what you are saying about the determinant of the matrix? What am I supposed to do with the $d$'s? Thanks. $\endgroup$ Nov 15 '13 at 23:16
2
$\begingroup$

Consider a vector whose $j$th component is \begin{align} & \phantom{={}}\text{constant}\cdot\int_{\mathbb R^n} \frac{1}{\sqrt{d_1\cdots d_n}} \exp\left(\frac{-1}{2} \sum_{k=1}^n d_k y_k^2 \right) y_j\,dy_1\cdots dy_n \\[12pt] & = c\int_{\mathbb R^n} \prod_{k=1}^n\left(\frac{1}{\sqrt{d_k}} \exp\left(\frac{-1}{2} d_ky_k^2\right)\right) y_j\,dy_1 \cdots dy_k \\[12pt] & = c\prod_{k=1}^n \int_{\mathbb R} \frac{1}{\sqrt{d_k}} \exp\left(\frac{-1}{2} d_ky_k^2\right) y_k\, dy_k. \end{align}

So it's reducible to integrals over $\mathbb R^1$, and if you're thinking about this particular problem, you probably know how to evaluate these particular integrals.

It can often happen that the purpose of diagonalizing a matrix is to reduce a problem involving a vector in $n$-space to $n$ problems involving scalars.

$\endgroup$
4
  • $\begingroup$ Thank you very much @MichaelHardy for your time. It seems to be right and convenient. I'll try to work on that and I'll upadate the result. So, what about your previous comment about the expected value? $\endgroup$ Nov 15 '13 at 23:26
  • 1
    $\begingroup$ The integral as written above evaluates to $0$. Somewhere you pulled out a separate constant term depending on $\mu$. If we'd had $(y_k+\mu_k)$, then the integral would evaluate to $\mu_k$ because the part before $y_k\,dy_k$ is a density function if the value of the constant is right. So you should end up with $\mu$ as the value of the integral you started with. $\endgroup$ Nov 15 '13 at 23:33
  • $\begingroup$ So, the original integral $I$ is equal to the mean vector $\mu$, right? That simple, right? And it can also be explained via the Gaussian distribution and it's expected value? $\endgroup$ Nov 15 '13 at 23:39
  • 1
    $\begingroup$ Correct. This is a way of proving that the expected value of a distribution with that density function is what its....expected....to be. Similarly if on integrates the $n\times n$ matrix $(\mathbf{x-\mu})(\mathbf{x-\mu})^T$ times that density function, one should get $\Sigma$. $\endgroup$ Nov 15 '13 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.