6
$\begingroup$

Let $X$, $Y$ and $Z$ be random variables.

  • Is it always possible to decompose $Y=U_1+U_2$ such that $\operatorname{Cov}(U_1,X\mid Z)=0$ and $\operatorname{Cov}(U_2,X\mid Z)=\operatorname{Cov}(U_2,X)$ i.e. the first term and $X$ are uncorrelated conditional on $Z$ and the second term and $X$ can be correlated independently of $Z$? I don't need explicit expressions for $U_1$ or $U_2$.

  • Can the covariances be replaced with independence (i.e. $U_1 \perp X\mid Z$ and $(U_2,X) \perp Z$)?

  • Are there any similar results?

  • What if $Y$ were replaced with $g(Y)=U_1+U_2$ for some function $g$?

I am interested in methods of how to go about solving such a problem and references. Thanks.

$\endgroup$
  • 2
    $\begingroup$ See book W. J. Krzanowski, Principles of Multivariate Analysis. It has the kind of materials you are asking for. $\endgroup$ – gopal Mar 14 '15 at 14:30
  • $\begingroup$ Thanks @gopal. Do you know the relevant chapters and sections? Unfortunately I don't have access to the book but I can try google the key words (I will also check the library). $\endgroup$ – user103828 Mar 14 '15 at 16:32
2
+100
$\begingroup$

(This CW is a slight simplification and extension of Michael's answer.)

Claim

$Y$ has a decomposition of the required form iff $\text{Cov}(X,Y|Z)$ is constant.

Proof

If:

Let $U_1=\mathbb{E}[Y|Z]$ and $U_2=Y-U_1$. Then $\text{Cov}(U_1, X|Z)=0$. $$ \begin{align} \text{Cov}(U_2, X) &= \text{Cov}(Y-\mathbb{E}[Y|Z], X)\\ &= \mathbb{E}[X(Y-\mathbb{E}[Y|Z])]\\ &= \mathbb{E}[XY-X\mathbb{E}[Y|Z]]\\ &= \mathbb{E}[\mathbb{E}[XY|Z] - \mathbb{E}[X|Z]\mathbb{E}[Y|Z]]\\ &= \mathbb{E}[\text{Cov}(X,Y|Z)]]\\ &=\text{Cov}(X,Y|Z)\\ &=\text{Cov}(X, U_1+U_2|Z)\\ &=\text{Cov}(X, U_2|Z) \end{align} $$

Onlf if:

Suppose we have a decomposition $Y=U_1+U_2$ satisfying the required conditions. $$\begin{align} \text{Cov}(U_2, X) &= \text{Cov}(U_2, X|Z)\\ &= \text{Cov}(Y - U_1, X|Z)\\ &= \text{Cov}(X, Y|Z) - \text{Cov}(U_1, X|Z)\\ &= \text{Cov}(X, Y|Z), \end{align} $$ so $\text{Cov}(X, Y|Z)$ is constant.

Example

Suppose $X\sim U(\{-1,0,1\})$, $Y=\mathbf{1}(X=1)$ and $Z=\mathbf{1}(X\neq 0)$, where $\mathbf{1}$ denotes the indicator function. This example doesn't satisfy the necessary condition, since $\text{Cov}(X,Y|Z)$ depends on $Z$.

$\endgroup$
  • $\begingroup$ Very nice. As a minor detail, this assumes $U_1$ is a pure function of $X$, although in general it could also depend on some external randomness. $\endgroup$ – Michael Mar 19 '15 at 19:50
  • $\begingroup$ Perhaps the general case is obtained if we interpret $U_1(1) \equiv E[U_1|X=1]$ and $U_1(-1)\equiv E[U_1|X=-1]$. $\endgroup$ – Michael Mar 19 '15 at 19:51
  • 1
    $\begingroup$ @Michael Thanks. I've simplified my answer using that idea. $\endgroup$ – Ben Derrett Mar 19 '15 at 20:48
  • 1
    $\begingroup$ I expanded on what you did in my answer below that gives a general necessary condition. $\endgroup$ – Michael Mar 20 '15 at 0:17
  • $\begingroup$ Elegant. This was a nice problem. Very clever thinking up $f(Z)=E[Y|Z]$ for a general sufficient condition. $\endgroup$ – Michael Mar 20 '15 at 22:52
2
$\begingroup$

Ben Derrett solves this question (he shows it is generally impossible). Here is a necessary condition for doing what you want:

Claim: A necessary condition is that $Cov(X,Y|Z)$ has no dependence on $Z$.

Proof: Suppose we have $U_1,U_2$ such that $U_1+U_2=Y$ and $Cov(U_1,X|Z)=0$ for all $Z$. Then $E[U_1X|Z] =E[U_1|Z]E[X|Z]$ and so:

\begin{align} Cov(U_2, X|Z) &=Cov(Y-U_1,X|Z) \\ &=E[(Y-U_1)X|Z] - E[Y-U_1|Z]E[X|Z]\\ &=E[YX|Z] - E[U_1X|Z] - E[Y|Z]E[X|Z]+E[U_1|Z]E[X|Z]\\ &=E[YX|Z] - E[Y|Z]E[X|Z]\\ &= Cov(X,Y|Z) \end{align} and hence we require $Cov(X,Y|Z)$ to have no dependence on $Z$.


Here is positive result about what can be done in this direction:

Given a random vector $(X,Y,Z)$, define $U_1= f(Z)$ for some real-valued function $f(z)$. Then:

\begin{align} XU_1 &= Xf(Z) \\ E[XU_1|Z] &= E[X f(Z)|Z] \\ &= f(Z)E[X|Z]\\ &= E[f(Z)|Z] E[X|Z]\\ &= E[U_1|Z]E[X|Z] \end{align}

and so indeed $Cov(U_1,X|Z) = 0$ for all $Z$. Hence, by the proof of the first claim, it also holds that $Cov(U_2, X|Z) = Cov(X,Y|Z)$.

This holds for all functions $f(z)$. If $Cov(X,Y|Z)$ does not depend on $Z$, sometimes we can choose $f(z)$ so that $Cov(X,Y|Z)=Cov(U_2, X)$, in which case all of your desired properties hold.

For example, suppose $Cov(X,Y|Z)=b$ for all $Z$. Let $f(z)=az$ for some real number $a$. Thus, $U_1=aZ$, $U_2=Y-aZ$, and we have: \begin{align} Cov(U_2,X) &= E[X(Y-aZ)] - E[X]E[Y-aZ]\\ &= E[XY] - aE[XZ] - E[X]E[Y]+aE[X]E[Z]\\ &= Cov(X,Y) - aCov(X,Z) \end{align} If $Cov(X,Z) \neq 0$ we can choose $a$ so that the above is equal to $b$, namely: $$ a = \frac{Cov(X,Y)-b}{Cov(X,Z)} $$ and all of your desired properties hold. So a sufficient condition is that $Cov(X,Y|Z)$ does not depend on $Z$, and $Cov(X,Z)\neq 0$.

$\endgroup$
  • 1
    $\begingroup$ Very nice! Using the bilinearity of the covariance reduces the proof of necessity to one line. Letting $f(Z)=\mathbb{E}[Y|Z]$ shows the condition is also sufficient. $\endgroup$ – Ben Derrett Mar 20 '15 at 8:42
  • $\begingroup$ @BenDerrett and Michael. Thanks for the solutions. This is precisely what I was looking for (sorry for the late reaction by me)... I'm torn to who to give the bounty. Is there a way to split the bounty? $\endgroup$ – user103828 Mar 20 '15 at 14:51
  • $\begingroup$ Try keeping the question open for a bit. Ben's sufficient condition is not clear to me...I would like to know if the necessary and sufficient conditions can be matched. You can give the bounty to Ben eventually. $\endgroup$ – Michael Mar 20 '15 at 14:53
  • $\begingroup$ @BenDerrett are you sure $f(Z)=E[Y|Z]$ is suffficient? We need to prove $Cov(Y-E[Y|Z], X)=Cov(X,Y|Z)$, which reduces to proving $E[XY] - E[XE[Y|Z]] = E[XY|Z] - E[X|Z]E[Y|Z]$. $\endgroup$ – Michael Mar 20 '15 at 15:06
  • $\begingroup$ The most general sufficient condition I can currently muster is if $Cov(X,Y|Z)$ does not depend on $Z$, and if there is a function $g(Z)$ such that $Cov(X, g(Z))\neq 0$, in which case we define $U_1=ag(Z)$ for a suitable $a$. $\endgroup$ – Michael Mar 20 '15 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.