# Finding the marginal distributions of a Gaussian mixture model. Is it the same as the Gaussian distribution?

Does the marginals of mixtures of Gaussians follow the properties of Gaussian distribution and the definition of marginalization? What I want to do is to obtain the marginal probability density function of each $$x_j$$, for all $$j=1,\ldots,n$$, where $$X=(x_1,\ldots,x_j,\ldots,x_n)^\top$$ in a Gaussian mixture model $$p(x)=\Sigma_{k=1}^Kw_kN(x|\mu_k,\Sigma_k)$$

For the Gaussian distribution I know from the Wikipedia's article to find the marginal distribution I can simple to this.: (And I've seen a lot of proof for this.)

To obtain the marginal distribution over a subset of multivariate normal random variables, one only needs to drop the irrelevant variables (the variables that one wants to marginalize out) from the mean vector and the covariance matrix.

I have seen that the marginals of Gaussian mixtures will also be Gaussian mixtures, but I haven't come across any mathematical proof to support this. Does this mean that, as I will do with the Gaussian distribution, I can just use the diagonal entries of the Sigma (of each component) for the variances of the random variables? And is there a proof for this?

If the covariance matrices $$\Sigma_k$$ are diagonal matrices, each component in the random vector $$X$$ also follows a mixture of univariate Gaussian distributions with the same mixing weights and the marginal variances are given by the diagonal elements of $$\Sigma_k$$. The reason for this is (the location parameter is set to zero to simplify the notation without loss of generality):
Lemma. If $$Z = (Z_1, \ldots, Z_n)'$$ is Gaussian random vector s.t. $$Z \sim N(0, \Sigma)$$, with $$\Sigma = \text{diag}(\sigma_1^2, \ldots, \sigma_n^2)$$, then $$\forall i\neq j$$, $$Z_i$$ and $$Z_j$$ are statistically independent and $$Z_j \sim N(0, \sigma_j^2)$$.
Proof. Both the independence and the marginal Gaussianity can be proved by noting that the joint density factors out into product of marginals: $$f_{Z}(z ; \Sigma) = (2\pi)^{-n/2} |\Sigma|^{-1/2}\exp\left(-\frac{1}{2} z'\Sigma^{-1}z \right) = \prod^n_{j=1}\varphi(z_j; \sigma_j^2), \ z \in \mathbb{R}^n, \ z_j \in \mathbb{R},$$ where $$\varphi(z_j; \sigma_j^2) = (2\pi)^{-1/2} (\sigma_j^2)^{-1/2} \exp\left(-\frac{1}{2} \frac{ z_j^2}{\sigma_j^2} \right).$$ The proof of the equalities above is straightforward by noting that $$|\Sigma|^{-1/2} = \prod^n_{j=1}(\sigma_j^2)^{-1/2}$$ and $$z'\Sigma^{-1}z = \sum^{n}_{j=1}z_j^2/\sigma_j^2$$. $$\square$$
Proof of the claim. Without loss of generality, suppose $$X$$ is a mixture of two $$n$$-variate Gaussian distributions $$N(0, \Sigma_1)$$ (with weight $$p \in [0,1]$$) and $$N(0, \Sigma_2)$$ with joint p.d.f. $$f_X(x; \Sigma_1, \Sigma_2, p) = pf_{Z}(x ; \Sigma_1) + (1-p)f_{Z}(x ; \Sigma_2), \ x \in \mathbb{R}^n.$$ If $$\Sigma_1$$ and $$\Sigma_2$$ are both diagonal such that $$\Sigma_k = \text{diag}(\sigma_{1,k}^2, \ldots, \sigma_{n,k}^2)$$, $$k = 1, 2$$, then by the preceding Lemma, $$f_X(x; \Sigma_1, \Sigma_2, p) = p\prod^n_{j=1}\varphi(x_j; \sigma_{j,1}^2)+ (1-p)\prod^n_{j=1}\varphi(x_j; \sigma_{j,2}^2).$$ Thus, for any $$j \in \{1, \ldots, n\}$$ the marginal density of the random variable $$X_j$$ is given by \begin{align} f_{X_j}(x_j) &= p\varphi(x_j; \sigma_{j,1}^2) \prod_{i\neq j} \underbrace{\int\varphi(x_i; \sigma_{i,1}^2) dx_i}_{=1} + (1-p)\varphi(x_j; \sigma_{j,2}^2)\prod_{i\neq j} \underbrace{\int\varphi(x_i; \sigma_{i,2}^2) dx_i}_{=1}\\ &= p\varphi(x_j; \sigma_{j,1}^2) +(1-p)\varphi(x_j; \sigma_{j,2}^2), \ x_j \in \mathbb{R} \end{align}. In other words, $$X_j$$ is a mixture of two univariate Gaussian distributions $$N(0, \sigma_{j,1}^2)$$ (with weight $$p$$) and $$N(0, \sigma_{j,2}^2)$$. $$\square$$