# Lower bound on additive non-gaussian noise under covariance constraint

Let $X_G \sim \mathcal{CN}(\mathbf{0},\mathbf{\Sigma}_X)$, $Z$ be some random vector with unknown distribution and non-singular covariance $\mathbb{E}\{ ZZ^H \} = \mathbf{\Sigma}_Z$. Let $X_g$ and $Z$ be uncorrelated but not necessarily be independent. Furthermore, let $$\tag{1} Y = X_G + Z.$$ I'm interested in the mutual information $$\tag{2} I(X_G;Y) = h(Y) - h(Y \vert X_G) = h(Y) - h( Z ) .$$

According to e.g. [1, Theorem 2], the differential entropy $h(Z)$ is upper bounded by the entropy of a complex Gaussian random vector $Z_G \sim \mathcal{CN}(\mathbf{0},\mathbf{\Sigma}_Z)$ with the same covariance $$\tag{3} h(Z) \leq h(Z_G) = \log \det \left( \pi e \mathbf{\Sigma}_Z \right),$$ because the complex Gaussian distribution has the largest entropy. Using this I obtain the following lower bound on (2) $$\tag{4} I(X_G;Y_G) = h(Y_G) - h(Z_G) \leq h(Y) - h(Z) = I(X_G;Y),$$ where $Y_G$ indicates that $Y$ is complex normal distributed in this case.

In contrast, the authors of [2, Lemma II.2] obtain the same lower bound (4), however, they require that $Z$ and $Z_G$ are both independent of $X_G$ (which they are not in my case). They use a different derivation which mainly relies on Jensen's inequality. However, I do not see where they make use of $Z$ and $Z_G$ being independent of $X_G$ in their derivation.

Also in [3, Theorem 1] the authors discuss a similar problem where $Z$ and $X_G$ are uncorrelated but not independent. However, the main step of their solution is also the Gaussian entropy upper bound (3). Furthermore, they also obtain the worst case covariance matrix $\mathbf{\Sigma}_Z^*$ (which I am not interested in).

In summary, I have the following questions

1. Are the lower bounds (3) and (4) valid?
2. Why does [2, Lemma II.2] require $Z$ and $Z_G$ both to be independent of $X_G$, which is also the reason for the "new" theorem in [3]?
• What does notation $Z/Z_G$ mean? Also, you cannot prove (4) by your approach since you have not considered how $h(Y)$ and $h(Y_G)$ are related (i.e., a Gaussian $Z$ results in increasing $h(Z)$ but also increases $h(Y)$). I believe the correct way to show (4) is to use the expansion $I(X_G;Y)=h(X_G)-h(X_G|Y)$ and note that the second term is maximized when $X_G$ and $Y$ are jointly Gaussian, which means a Gaussian $Z$. Commented Aug 9, 2017 at 13:18
• Thank you for your comment. With $Z/Z_G$ I mean $Z$ and $Z_G$ should both be independent of $X_G$. I'll edit it. You are right, I didn't think about increasing $h(Y)$ aswell. I guess $h(X_G \vert Y)$ is maximized if $X_G$ and $Y$ are jointly Gaussian for the same reasons as in (3)? So I don't need $X_G$ and $Z$ to be independent?
– Enzo
Commented Aug 9, 2017 at 14:01
• Yes, that's the argument for proving (4). Note that since $X_G$ and $Z$ are uncorrelated by assumption, they are also independent in the case they are selected as jointly Gaussian. Commented Aug 9, 2017 at 14:16

Eq (2) is wrong. The last term should be $h(Z\mid X_G)$. In general
$$h(Y|X_G)=h(X_G + Z|X_G)=h(Z\mid X_G)$$ You can equate this to $h(Z)$ only if $Z,X_G$ are independent.
• Thank you very much for your answer. You are completely right, my mistake. However, if I follow @Stelios' approach, choosing $Z$ to be normal distributed and hence maximizing $h(X_G | Y)$, $Z$ and $X_G$ will be independent and it checks out? One again, thank you!