6
$\begingroup$

Let $X$, $Y$ and $Z$ be random variables.

  • Is it always possible to decompose $Y=U_1+U_2$ such that $\operatorname{Cov}(U_1,X\mid Z)=0$ and $\operatorname{Cov}(U_2,X\mid Z)=\operatorname{Cov}(U_2,X)$ i.e. the first term and $X$ are uncorrelated conditional on $Z$ and the second term and $X$ can be correlated independently of $Z$? I don't need explicit expressions for $U_1$ or $U_2$.

  • Can the covariances be replaced with independence (i.e. $U_1 \perp X\mid Z$ and $(U_2,X) \perp Z$)?

  • Are there any similar results?

  • What if $Y$ were replaced with $g(Y)=U_1+U_2$ for some function $g$?

I am interested in methods of how to go about solving such a problem and references. Thanks.

$\endgroup$
2
  • 2
    $\begingroup$ See book W. J. Krzanowski, Principles of Multivariate Analysis. It has the kind of materials you are asking for. $\endgroup$
    – gopal
    Mar 14, 2015 at 14:30
  • $\begingroup$ Thanks @gopal. Do you know the relevant chapters and sections? Unfortunately I don't have access to the book but I can try google the key words (I will also check the library). $\endgroup$
    – user103828
    Mar 14, 2015 at 16:32

2 Answers 2

2
+100
$\begingroup$

(This CW is a slight simplification and extension of Michael's answer.)

Claim

$Y$ has a decomposition of the required form iff $\text{Cov}(X,Y|Z)$ is constant.

Proof

If:

Let $U_1=\mathbb{E}[Y|Z]$ and $U_2=Y-U_1$. Then $\text{Cov}(U_1, X|Z)=0$. $$ \begin{align} \text{Cov}(U_2, X) &= \text{Cov}(Y-\mathbb{E}[Y|Z], X)\\ &= \mathbb{E}[X(Y-\mathbb{E}[Y|Z])]\\ &= \mathbb{E}[XY-X\mathbb{E}[Y|Z]]\\ &= \mathbb{E}[\mathbb{E}[XY|Z] - \mathbb{E}[X|Z]\mathbb{E}[Y|Z]]\\ &= \mathbb{E}[\text{Cov}(X,Y|Z)]]\\ &=\text{Cov}(X,Y|Z)\\ &=\text{Cov}(X, U_1+U_2|Z)\\ &=\text{Cov}(X, U_2|Z) \end{align} $$

Onlf if:

Suppose we have a decomposition $Y=U_1+U_2$ satisfying the required conditions. $$\begin{align} \text{Cov}(U_2, X) &= \text{Cov}(U_2, X|Z)\\ &= \text{Cov}(Y - U_1, X|Z)\\ &= \text{Cov}(X, Y|Z) - \text{Cov}(U_1, X|Z)\\ &= \text{Cov}(X, Y|Z), \end{align} $$ so $\text{Cov}(X, Y|Z)$ is constant.

Example

Suppose $X\sim U(\{-1,0,1\})$, $Y=\mathbf{1}(X=1)$ and $Z=\mathbf{1}(X\neq 0)$, where $\mathbf{1}$ denotes the indicator function. This example doesn't satisfy the necessary condition, since $\text{Cov}(X,Y|Z)$ depends on $Z$.

$\endgroup$
6
  • $\begingroup$ Very nice. As a minor detail, this assumes $U_1$ is a pure function of $X$, although in general it could also depend on some external randomness. $\endgroup$
    – Michael
    Mar 19, 2015 at 19:50
  • $\begingroup$ Perhaps the general case is obtained if we interpret $U_1(1) \equiv E[U_1|X=1]$ and $U_1(-1)\equiv E[U_1|X=-1]$. $\endgroup$
    – Michael
    Mar 19, 2015 at 19:51
  • 1
    $\begingroup$ @Michael Thanks. I've simplified my answer using that idea. $\endgroup$ Mar 19, 2015 at 20:48
  • 1
    $\begingroup$ I expanded on what you did in my answer below that gives a general necessary condition. $\endgroup$
    – Michael
    Mar 20, 2015 at 0:17
  • $\begingroup$ Elegant. This was a nice problem. Very clever thinking up $f(Z)=E[Y|Z]$ for a general sufficient condition. $\endgroup$
    – Michael
    Mar 20, 2015 at 22:52
2
$\begingroup$

Ben Derrett solves this question (he shows it is generally impossible). Here is a necessary condition for doing what you want:

Claim: A necessary condition is that $Cov(X,Y|Z)$ has no dependence on $Z$.

Proof: Suppose we have $U_1,U_2$ such that $U_1+U_2=Y$ and $Cov(U_1,X|Z)=0$ for all $Z$. Then $E[U_1X|Z] =E[U_1|Z]E[X|Z]$ and so:

\begin{align} Cov(U_2, X|Z) &=Cov(Y-U_1,X|Z) \\ &=E[(Y-U_1)X|Z] - E[Y-U_1|Z]E[X|Z]\\ &=E[YX|Z] - E[U_1X|Z] - E[Y|Z]E[X|Z]+E[U_1|Z]E[X|Z]\\ &=E[YX|Z] - E[Y|Z]E[X|Z]\\ &= Cov(X,Y|Z) \end{align} and hence we require $Cov(X,Y|Z)$ to have no dependence on $Z$.


Here is positive result about what can be done in this direction:

Given a random vector $(X,Y,Z)$, define $U_1= f(Z)$ for some real-valued function $f(z)$. Then:

\begin{align} XU_1 &= Xf(Z) \\ E[XU_1|Z] &= E[X f(Z)|Z] \\ &= f(Z)E[X|Z]\\ &= E[f(Z)|Z] E[X|Z]\\ &= E[U_1|Z]E[X|Z] \end{align}

and so indeed $Cov(U_1,X|Z) = 0$ for all $Z$. Hence, by the proof of the first claim, it also holds that $Cov(U_2, X|Z) = Cov(X,Y|Z)$.

This holds for all functions $f(z)$. If $Cov(X,Y|Z)$ does not depend on $Z$, sometimes we can choose $f(z)$ so that $Cov(X,Y|Z)=Cov(U_2, X)$, in which case all of your desired properties hold.

For example, suppose $Cov(X,Y|Z)=b$ for all $Z$. Let $f(z)=az$ for some real number $a$. Thus, $U_1=aZ$, $U_2=Y-aZ$, and we have: \begin{align} Cov(U_2,X) &= E[X(Y-aZ)] - E[X]E[Y-aZ]\\ &= E[XY] - aE[XZ] - E[X]E[Y]+aE[X]E[Z]\\ &= Cov(X,Y) - aCov(X,Z) \end{align} If $Cov(X,Z) \neq 0$ we can choose $a$ so that the above is equal to $b$, namely: $$ a = \frac{Cov(X,Y)-b}{Cov(X,Z)} $$ and all of your desired properties hold. So a sufficient condition is that $Cov(X,Y|Z)$ does not depend on $Z$, and $Cov(X,Z)\neq 0$.

$\endgroup$
12
  • 1
    $\begingroup$ Very nice! Using the bilinearity of the covariance reduces the proof of necessity to one line. Letting $f(Z)=\mathbb{E}[Y|Z]$ shows the condition is also sufficient. $\endgroup$ Mar 20, 2015 at 8:42
  • $\begingroup$ @BenDerrett and Michael. Thanks for the solutions. This is precisely what I was looking for (sorry for the late reaction by me)... I'm torn to who to give the bounty. Is there a way to split the bounty? $\endgroup$
    – user103828
    Mar 20, 2015 at 14:51
  • $\begingroup$ Try keeping the question open for a bit. Ben's sufficient condition is not clear to me...I would like to know if the necessary and sufficient conditions can be matched. You can give the bounty to Ben eventually. $\endgroup$
    – Michael
    Mar 20, 2015 at 14:53
  • $\begingroup$ @BenDerrett are you sure $f(Z)=E[Y|Z]$ is suffficient? We need to prove $Cov(Y-E[Y|Z], X)=Cov(X,Y|Z)$, which reduces to proving $E[XY] - E[XE[Y|Z]] = E[XY|Z] - E[X|Z]E[Y|Z]$. $\endgroup$
    – Michael
    Mar 20, 2015 at 15:06
  • $\begingroup$ The most general sufficient condition I can currently muster is if $Cov(X,Y|Z)$ does not depend on $Z$, and if there is a function $g(Z)$ such that $Cov(X, g(Z))\neq 0$, in which case we define $U_1=ag(Z)$ for a suitable $a$. $\endgroup$
    – Michael
    Mar 20, 2015 at 15:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .