1
$\begingroup$

Let $\theta\sim\mathcal{N}(\bar{\theta},1/\tau_\theta)$ be a normally distributed random variable, $\varepsilon\sim\mathcal{N}(0,1/\tau_\varepsilon)$ be a normally distributed noise term independent from $\theta$, and let $P$ be an affine function of $\theta$ and $\varepsilon$, i.e., $P(\theta,\varepsilon)=a+b\theta+c\varepsilon$ with $b,c\neq 0$.

I want to determine the conditional variance $Var(\theta+P|P=p)$.

I know how to determine the conditional variance $Var(\theta|P=p)$, which should be $Var(\theta|P)=b^2/\tau_\theta+c^2/\tau_\varepsilon$, since $\theta$ and $\varepsilon$ are uncorrelated. I am not entirely sure about $Var(\theta+P|P=p)$ though, since $P$ and $\theta$ are not uncorrelated.

Here is a guess: \begin{equation} Var(\theta+P|P=p)=Var(\theta|P=p)+Var(P|P=p)+2Cov(\theta,P|P=p)\\ =Var(\theta|P=p)+2Cov(\theta,P|P=p) \end{equation} Now I am not sure; since conditional on $P=p$, $P$ does not vary, the covariance term $$2Cov(\theta,P|P=p)=2E[(\theta-E[\theta|P=p])(P-E[P|P=p])|P=p]$$ should be zero, which implies $Var(\theta+P|P=p)=Var(\theta|P=p)$.

Another approach that seems plausible would be \begin{equation} Var(\theta+P|P=p)=Var(\theta+a+b\theta+c\varepsilon|P=p)\\ =Var(\theta(1+b)+c\varepsilon|P=p)\\ \neq Var(\theta|P=p) \end{equation}

Which is it, or is it something else entirely? If it is the latter, what is $Var(\theta(1+b)+c\varepsilon|P=p)$ explicitly? Thanks!

$\endgroup$

2 Answers 2

2
$\begingroup$

Your first derivation leading to $Var(\theta+P|P=p)=Var(\theta|P=p)$ is correct. Your second derivation is also correct except for the final statement $$Var(\theta+P|P=p) = Var(\theta(1+b)+c\varepsilon|P=p) \implies Var(\theta+P|P=p) \neq Var(\theta|P=p)$$ They could be equal, although they don't appear to be. In fact they are. Here is how to reason it out. You have a relationship between $p, \varepsilon$ and $\theta$: $$P(\theta,\varepsilon)=a+b\theta+c\varepsilon$$ Since $a$ and $b$ are constants, with knowledge of $p$, you also know what the value of the sum $b\theta+c\varepsilon$ is. Now $$Var(\theta(1+b)+c\varepsilon|P=p) = Var(\theta+(b\theta+c\varepsilon)|b\theta+c\varepsilon) = Var(\theta | b\theta+c\varepsilon) = Var(\theta | p)$$ as desired.

$\endgroup$
2
  • $\begingroup$ Great, thank you, very good answer! Do you have an intuition why the conditional variance of $\theta+P$ is equal to the conditional variance of $\theta$? Is it that, by knowing $P=p$, you know the part of the sum $(b\theta+c\varepsilon)$ that "amplifies" the variation of $\theta$ (if $b>0$)? $\endgroup$
    – Nameless
    Aug 11, 2015 at 1:28
  • 1
    $\begingroup$ @Nameless - I said that slightly wrong: $p$ does give you information about the variance of $\theta + P$, but it is the same information that knowing $b\theta + c\varepsilon$ tells you about it. $\endgroup$
    – muaddib
    Aug 11, 2015 at 1:51
1
$\begingroup$

It is not true that $Var(\theta|P)=b^2/\tau_\theta+c^2/\tau_\varepsilon$ in these settings. Take, for example, $\epsilon\sim N(0,1)$, $P\{v=-1\}=P\{v=1\}=1/2$ (independent of $\epsilon$), and $\theta=v\epsilon$. Then

$$P\{\theta\le t\}=\frac{1}{2}P\{\epsilon\le t\}+\frac{1}{2}P\{\epsilon\ge -t\}=\Phi(t)$$

so that $\theta\sim N(0,1)$ and $Cov(\epsilon, \theta)=0$. However, $\theta$ is not independent of $\epsilon$.

Assume that $a=0$, $b=c=1$. Then $Var(\theta|P=p)=0$ for any $p\ne 0$ ($P\ne 0\Rightarrow v=1$ so that $P=2\epsilon$ and $\theta=\epsilon$).


You need to specify the joint distribution of $(\theta,\epsilon)$ either explicitly or by assuming that $\theta$ and $\epsilon$ are independent (not just uncorrelated).


In any case,

$$Var(\theta+P|P)=\mathbb{E}[(\theta+P)^2|P]-(\mathbb{E}[\theta+P|P])^2$$ $$=\mathbb{E}[\theta^2|P]-(\mathbb{E}[\theta|P])^2=Var(\theta|P)$$

$\endgroup$
1
  • $\begingroup$ Thank you for this elegant example, which nicely illustrates the pitfalls of uncorrelated$\neq$independent. I meant to assume independence between the two normal distributions. $\endgroup$
    – Nameless
    Aug 11, 2015 at 1:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .