Consider a random variable $X$ and random vector $\mathbf Y$. The law of iterated variance states that $$ \mbox{Var}(X) = E\left\{\mbox{Var}(X|\mathbf Y)\right\} + \mbox{Var}\left\{E(X|\mathbf Y)\right\}. $$ Immediately we have that $\mbox{Var}(X|\mathbf Y = \mathbf y)$ is, on average less than $\mbox{Var}(X)$, however this does not necessarily hold uniformly in $\mathbf y$. An obvious example where it does not is where $X | Y = y \sim N(0, y)$ where $y = 1$ with probability $\frac 1 2$ and $y = 10$ with probability $\frac 1 2$.

I'm interested in when I can say that $\mbox{Var}(X) > \mbox{Var}(X \mid \mathbf Y = \mathbf y)$ for all values of $\mathbf y$. This corresponds to, effectively, saying that observing $\mathbf y$ always improves our knowledge of $\mathbf X$. $\mathbf X$ is more tightly concentrated, no matter which $\mathbf y$ we observe, rather than just being more tightly concentrated on average.

In particular, I'd like some sort of condition that can be expressed in terms of easily checked properties of the distribution of $X$ and the distribution of $\mathbf Y | X$ (something nice, like log-concavity of the density of $X$ or $\mathbf Y | X$, which fails for the mixture above). Checking the condition $\mbox{Var}(X) > \mbox{Var}(X|\mathbf Y)$ is equivalent to checking a convexity condition for an optimization problem, while $X|Y$ is intractable.

  • $\begingroup$ Not that I have an answer, but did you mean X|Y = x ~ N(0,y) perhaps? $\endgroup$ – Avraham Aug 29 '13 at 4:55
  • $\begingroup$ @Avraham the notation means "the distribution of $X$ given that $Y =y$." $\endgroup$ – guy Aug 29 '13 at 13:14
  • $\begingroup$ Is there something wrong with the question? Someone downvoted and I don't understand why. $\endgroup$ – guy Aug 29 '13 at 13:15
  • $\begingroup$ I understand the notation :). However, as you yourself say, it is $x$ which is depending on $y$ in that it is normally distributed with mean 0 and variance $y$. But you wrote $X | Y = \mathbf{y} \sim N(0, y)$ which means y is distributed with mean 0 and variance itself, which I do not think is what you meant. $\endgroup$ – Avraham Aug 29 '13 at 15:42
  • $\begingroup$ @Avraham the notation I used is standard for writing conditional distributions in my field, the $\sim$ is associated with $X$, not $y$. $\endgroup$ – guy Aug 29 '13 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.