0
$\begingroup$

One can frequently read that the product of two multivariate Gaussian pdfs, $f_1(x)$*$f_2(x)$, is itself a Gaussian function, with parameters as defined for example in:

http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf (section 8.1.8 page 42)

But what if the parameters (mean vector, covariance matrix) for $f_2$ are not given for all elements of $x$? I.e. if the mean vector of $f_2$ contains values only for some of the random elements of $x$, and the $f_2$ covariance matrix - which is here assumed to be diagonal - also contains only values for this subset of the elements of $x$? Is the product then still a Gaussian and how can the parameters be calculated?

$\endgroup$
5
  • 1
    $\begingroup$ Hint: $f_{X,Y}{x,y} = f_X(x)f_Y(y)$ for independent random variables (or vectors). For multivariate random variables, the mean vector is the concatenation of the constituent mean vectors while the covariance matrix is $$\Sigma = \left[\begin{matrix}\Sigma_1&\mathbf 0\\\mathbf 0& \Sigma_2\end{matrix}\right]$$ $\endgroup$ Apr 20, 2015 at 19:09
  • $\begingroup$ @Dilip Sarwate Thank you Dilip - yes, I think this will help... in the meantime, I found a way for the underlying problem so both densities' covariance matrices and mean vectors contain expressions for all the elements in the vector - still working on this, though. Hope I can come back with more clarification, soon. $\endgroup$
    – user70160
    Apr 21, 2015 at 1:46
  • $\begingroup$ @Dilip Sarwate thanks again - I think I got what you are saying. I could partition the vector $x$ into two parts, where one part, $x_a$ includes all those elements for which both $f_1$ and $f_2$ have parameters, and $x_b$ contains all the other elements. And then multiply $f_1(x_a)$ and $f_2(x_a)$ to get f(x_a). This I would then have to concatenate with $f_1(x_b)$ to get $f(x) = f_1(x)f_2(x)$. Is that what you had in mind? $\endgroup$
    – user70160
    May 2, 2015 at 0:28
  • $\begingroup$ Your notation is horrendous and adds to the overall confusion. When you write $f(x) = f_1(x)f_2(x)$, the three occurrences of $x$ represent three different things. You are thus violating a basic convention in mathematics that a symbol should mean the same thing everywhere it occurs in an expression or formula. $\endgroup$ May 2, 2015 at 3:39
  • $\begingroup$ @Dilip Sarwate - thanks for your feedback! however, I'm not sure if the notation here is that wrong. Here x is supposed to be the same vector/set of variables, but the densities are different. This was actually pointed out to me in an answer to an earlier question: please have a look at the answer I got there: math.stackexchange.com/questions/1221058/product-of-densities. The references given in the question also use similar notation. However, I'm not ruling out that I have misunderstood something. What do you think? $\endgroup$
    – user70160
    May 2, 2015 at 3:48

1 Answer 1

1
$\begingroup$

One can frequently read that the product of two multivariate Gaussian pdfs, $f_1(x)$*$f_2(x)$, is itself a Gaussian,

This is an incorrect statement no matter how many times or in how many different places you have read it; indeed it does not hold even for univariate Gaussian random variables. If the link you have provided does indeed say exactly what you have written, then I suggest that you apply the instructions given by Dorothy Parker in a famous book review: "This is not a book to be put down lightly; it should be thrown with great force."

What is true is that if $X$ and $Y$ are independent (univariate) Gaussian random variables with densities $f_X(x)$ and $f_Y(y)$ respectively, then the product $f_X(x)f_Y(y)$ equals the joint density function $f_{X,Y}(x,y)$. Note that the joint density is a function of two real variables which have different names $x$ and $y$, and it is not claimed that $f_X(x)f_Y(x)$ is a Gaussian density (note the lack of $y$).

More generally,if $\mathbf X = (X_1,X_2,\ldots, X_m)$ and $\mathbf Y = (Y_1, Y_2, \ldots, Y_n)$ are random vectors that are independent of each other and $\mathbf X$ and $\mathbf Y$ have multivariate Gaussian densities $$\begin{align} f_{\mathbf X}(\mathbf x) &= \frac{1}{(2\pi)^{m/2}\sqrt{|\operatorname{det}(\Sigma_X)|}}\exp\left(-\frac 12(\mathbf x - \mathbf m_X)\Sigma_X^{-1} (\mathbf x - \mathbf m_X)^T\right)\\ f_{\mathbf Y}(\mathbf y) &= \frac{1}{(2\pi)^{n/2}\sqrt{|\operatorname{det}(\Sigma_Y)|}}\exp\left(-\frac 12(\mathbf y - \mathbf m_Y)\Sigma_Y^{-1} (\mathbf y - \mathbf m_Y)^T\right) \end{align}$$ where $\Sigma_X$ and $\Sigma_Y$ are $m\times m$ and $n \times n$ covariance matrices, and $\mathbf m_X$ and $\mathbf m_Y$ are the mean vectors, of $\mathbf X$ and $\mathbf Y$ respectively. Note that these densities are functions of $m$ and $n$ real variables $(x_1, x_2, \ldots, x_m)$ and $(y_1, y_2,\ldots, y_n)$ respectively. Then $$f_{\mathbf X, \mathbf Y}\big ((\mathbf x, \mathbf y)\big) = f_{\mathbf X}(\mathbf x)f_{\mathbf Y}(\mathbf y)$$ where $(\mathbf x, \mathbf y) = (x_1, x_2, \ldots, x_m, y_1, y_2,\ldots, y_n)$ is a vector of $m+n$ real numbers. Plugging and chugging, we have that $$\begin{align} f_{\mathbf X, \mathbf Y}\big ((\mathbf x, \mathbf y)\big) &= \frac{\exp\left(-\frac 12(\mathbf x - \mathbf m_X)\Sigma_X^{-1} (\mathbf x - \mathbf m_X)^T-\frac 12(\mathbf y - \mathbf m_Y)\Sigma_Y^{-1} (\mathbf y - \mathbf m_Y)^T\right)}{(2\pi)^{(m+n)/2} \sqrt{|\operatorname{det}(\Sigma_X)|\cdot|\operatorname{det}(\Sigma_Y)|}}\tag{1}\\ &= \frac{\exp\left(-\frac 12(\mathbf z - \mathbf m)\Sigma^{-1} (\mathbf z - \mathbf m)^T\right)}{(2\pi)^{(m+n)/2} \sqrt{|\operatorname{det}(\Sigma)|}}\tag{2} \end{align}$$ where $\mathbf z = (\mathbf x, \mathbf y) = (x_1, x_2, \ldots, x_m, y_1, \ldots, y_n)$, $\mathbf m = (\mathbf m_X,\mathbf m_Y)$ is the mean vector of the $(m+n)$-dimensional multivariate Gaussian vector $\mathbf Z = (\mathbf X, \mathbf Y)$, and $$\Sigma = \left[\begin{matrix}\Sigma_X&\mathbf 0\\\mathbf 0& \Sigma_Y\end{matrix}\right]\tag{3}$$ is the $(m+n)\times (m+n)$ covariance matrix. The derivations are greatly simplified by observing the block structure $(3)$ of the covariance matrix.

$\endgroup$
4
  • $\begingroup$ Thank you for your effort. Would you mind sparing a moment to have a brief look at section 8.1.8 page 41 of "The Matrix Cookbook"? available here: imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/imm3274.pdf and maybe also at page 200 of this source: gaussianprocess.org/gpml/chapters/RWA.pdf section products at the bottom, and further maybe here: tina-vision.net/docs/memos/2003-003.pdf. This were for me the most helpful sources on that topic. $\endgroup$
    – user70160
    May 3, 2015 at 4:54
  • $\begingroup$ An example where this property is used is Bayesian estimation: see page 11 of the following: citeseerx.ist.psu.edu/viewdoc/…. I understand what you have written in your answer, and your initial comment - although referring to densities for two different variables, was helpful and brought me on the right track to solve the problem behind the original question, and I upvoted it. $\endgroup$
    – user70160
    May 3, 2015 at 5:03
  • $\begingroup$ I am trying to find my misunderstanding, but I am familiar with the property you are describing for independent variables/vectors, and do not see a contradiction to the statement you claim is wrong. I apologize if I have used improper notation and wording, the matter is still relatively new for me. I think the best would be if you raise your point here: math.stackexchange.com/questions/1221058/product-of-densities. there is an (imho) excellent answer by grand_chat, which helped me greatly to understand the cited sources. My own intial surprise is well documented there. $\endgroup$
    – user70160
    May 3, 2015 at 5:09
  • $\begingroup$ maybe the distinction between "Gaussian" and "Gaussian pdf" is the crucial point here? the quoted sources state, the product is a Gaussian, but it is not a pdf (not normalized). In other words, it is proportional to a Gaussian pdf, and by multiplying with a normalizing constant becomes a Gaussian pdf. $\endgroup$
    – user70160
    May 4, 2015 at 1:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .