Suppose I know that, for two random variables $X,Y$, we have $$Cov(X,Y)\neq 0.$$ What happens if we take a monotone transformation of $X$; will the inequality persist? That is, say $f(.)$ is a strictly increasing (not necessarily linear) function, then does it follow that $$Cov(f(X),Y)\neq 0?$$ Since covariance depicts linear relationships, this would only follow if $f$ were linear, right? In that case, under which assumptions on the relation of $X,Y$ can we say that it does follow for nonlinear transformations $f$? (If it helps, I am currently looking at the case where $f$ is the cdf of the standard normal $\Phi$.)


(I have edited the question changing $f$ to $g$ in order to keep the symbol $f$ for probability density functions). First, indeed, when the transformation is linear(affine actually), the covariance involving the transformation will continue to be non-zero. For non-linear transformations you cannot obtain very specific conditions. My approach is the following:

For compactness, set $Z=g\left(X\right)$. Then you require the conditions under which $$\operatorname{Cov}(Z,Y)\neq 0? \;\Rightarrow\; E\left(ZY\right)\,-\,E\left(Z\right)E\left(Y\right)\, \neq0?\qquad [1]$$

Using Integrals to express the expected values, we require (assuming certain regularity conditions) $$\int_{D_Z} \int_{D_Y} zyf_{Z,Y}\left(z,y\right)\,dy\,dz \;-\;\left(\int_{D_Z}zf_{Z}\left(z\right)dz\right) \left(\int_{D_Y} yf_{Y}\left(y\right)\,dy\right)\neq0?$$

$$\Rightarrow \int_{D_Z} \int_{D_Y} zyf_{Z,Y}\left(z,y\right)\,dy\,dz \;-\;\int_{D_Z} \int_{D_Y} zyf_{Z}\left(z\right)f_{Y}\left(y\right)\,dy\,dz\neq0?$$

$$\Rightarrow \int_{D_Z} \int_{D_Y} zy\left[f_{Z,Y}\left(z,y\right) \;-\;f_{Z}\left(z\right)f_{Y}\left(y\right)\right]\,dy\,dz\neq0?\qquad[2]$$

where the $f()$ functions are joint or marginal probability density functions, and $D$ denotes the support. Now define $\omega \equiv(z,y)$ and note that $zy\;$,$\,z\;$,$\;y\;$, can all be derived as linear transformations of $\omega$, say $zy=\tau_0(\omega)\;$,$\,z=\tau_1(\omega)\;$,$\;y=\tau_2(\omega)\;$. For example $z=\omega[1\; 0]'$, the prime denoting the transpose. Also define $C\equiv D_Z\times D_Y$. Then we can compact even further by writing $$\cdots\Rightarrow \int_C h\left[\omega,\tau_0(\omega),\tau_1(\omega),\tau_2(\omega)\right]d\omega\neq0?\qquad[3]$$

Why all this trouble? Because now it is clear that what you are asking is the requirements under which a double definite integral of a function of its variable does not equal zero when integrated over the whole domain of its variable. Reversely, we want to exclude the conditions under which this integral equals zero. First since $\operatorname{Cov}(X,Y)\neq 0\;$, it implies that the random variables $X$ and $Y$ are not independent. Therefore we have $f_{Z,Y}\left(z,y\right) \;\neq\;f_X \left(x\right)f_Y \left(y\right)$ . But this does not guarantee that the double integral will be non-zero. It may still equal zero if the integrand in eq.[2] is an odd function w.r.t. to at least one of $z$ or $y$, and if the domain w.r.t to this variable is symmetric around the origin (which will hold if for example $D= \Bbb R$ - extended). Assume that indeed $C=\Bbb R\times\Bbb R$ extended. Therefore we must guarantee that the integrand $zy\left[f_{Z,Y}\left(z,y\right) \;-\;f_{Z}\left(z\right)f_{Y}\left(y\right)\right]$ is not an odd function w.r.t. to either $z$ or $y$ separately. Now if you work the integrand, substituting $-z$ for $z$ etc, you will be able to reduce the condition into the following: "The non-linear transformation should not be such as to make both $f_{Z,Y}\left(z,y\right) \;$ and $f_{Z}\left(z\right)$ even functions of $z$." For $Y$ this could be stated as "If the pdf of $Y$ is an even function, then the non-linear transformation should not be such as to make $f_{Z,Y}\left(z,y\right) \;$ an even function of $y$.". Obviously this is distribution-specific (or at least distribution family- specific) and you cannot go more specific than that. If it looks disappointingly general, imagine a true jungle. Then independence is a straight clear path through this jungle, and linear dependence is an elliptical clear path approaching asymptotically the straight path of independence. The rest of the jungle is non-linear dependence. $$ \text{}$$ Afterthought: A less "mathematical" and more "statistical" condition (although not necessarily more useful) would be "The non-linear transformation must be such as to not render the variables $Z$ and $Y$ sub-independent". In a nutshell, sub-independent random variables are dependent but uncorrelated: their dependence is "perfectly non-linear".

  • $\begingroup$ Thanks for your answer. I see how the integrand in [2] being an odd function will make the covariance zero if we integrate over the reals. If I rewrite [2] for the odd $z$ part: $$\int_{-\infty}^0 f_{Z,Y}(z)-f_Z(z)f_y(z) dz+\int_{0}^\infty f_{Z,Y}(z)-f_Z(z)f_Y(z) dz$$ so $$\Leftrightarrow \int_0^\infty f_{Z,Y}(-z)-f_Z(-z)f_y(-z) +f_{Z,Y}(z)-f_Z(z)f_Y(z) dz\neq0.$$ But that means the transformation should not make both $f_{Z,Y}$ and $f_Z$ an odd function of $Z$, right? You write it should not make them an even function. $\endgroup$
    – Nameless
    Jul 30 '13 at 20:50
  • $\begingroup$ Don't forget the "z" outside the brackets! $\endgroup$ Jul 30 '13 at 21:06
  • $\begingroup$ Yes, you're right, they have to be even functions. But am I right that if both $X,Y$ are positive, then the "even function condition" is not applicable? Both of you provided sufficient conditions and I tossed a virtual coin to determine who gets the accept. You won. $\endgroup$
    – Nameless
    Aug 2 '13 at 21:46
  • $\begingroup$ Luck is always welcome, although not chased. Indeed, the property of "evenness" or "oddness" can only be defined for functions whose domain includes both positive and negative values. If $X,Y$ have positive range => the domain of the densities will be only in the positive orthant. $\endgroup$ Aug 2 '13 at 22:46

Assume that $\mathrm{Cov}(X,Y)\ne0$ and consider some increasing function $g$ (you say that the case of interest is when $g=\Phi$ the standard normal CDF, which is increasing). The question is to find some conditions on $(X,Y)$ implying that $\mathrm{Cov}(g(X),Y)\ne0$, presumably for every such $g$.

Here is a sufficient condition: one knows that $E[Y\mid X]=h(X)$ for some function $h$ depending only on the joint distribution of $(X,Y)$. Then:

If $h$ is increasing or if $h$ is decreasing, then $\mathrm{Cov}(g(X),Y)\ne0$.

To prove this, assume without loss of generality that $h$ is increasing, note that $\mathrm{Cov}(g(X),Y)$ is also $\mathrm{Cov}(g(X),h(X))=E[g(X)h(X)]-E[g(X)]E[h(X)]$, and that for nondecreasing functions $g$ and $h$, $E[g(X)h(X)]\geqslant E[g(X)]E[h(X)]$ with equality if and only if $g$ or $h$ is almost surely constant on $D$ such that $P[X\in D]=1$. Here $g$ and $h$ are increasing everywhere hence $E[g(X)h(X)]\gt E[g(X)]E[h(X)]$ except if $X$ is almost surely constant but then $\mathrm{Cov}(X,Y)=0$ hence this does not happen and the claim is proved.

Of course, the partial result above is only useful when one can compute $E[Y\mid X]=h(X)$ easily. Here are two remarks in this direction. First, when $(X,Y)$ has a density, $$ h(x)=f_X(x)^{-1}\int_{-\infty}^{+\infty}yf_{(X,Y)}(x,y)\mathrm dy, $$ hence one should be able to check the monotony of $h$ in some specific situations. Second, when $(X,Y)$ is normal, $h(x)=ax+b$ where $a=\mathrm{Cov}(X,Y)\sqrt{\mathrm{var}(Y)/\mathrm{var}(X)}$ hence $h$ is increasing if and only if $\mathrm{Cov}(X,Y)\gt0$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.