1
$\begingroup$

Suppose $X_1,X_2,...,X_N$ are i.i.d random variables. I want to calculate the conditional expectation $$ \mathbb{E}[X_i|\sum_{k=1}^N\alpha_k X_k]. $$

For now, my idea is as the following:

Denote $Y_k=\alpha_kX_k$ and let $\pi=(\pi_1,...,\pi_N)$ be any permutation of $(1,...,N)$. It follows from the pairwise independence of $(Y_1,...,Y_N)$ that $Y_1,...,Y_N$ are exchangeable, i.e., for any two permutations $\pi$ and $\pi'$, $(Y_{\pi_1},...,Y_{\pi_N})$ and $(Y_{\pi_1'},...,Y_{\pi_N'})$ share the same joint distribution. Thus, function $f(Y_{\pi_1},...,Y_{\pi_N})$ and $f(Y_{\pi_1'},...,Y_{\pi_N'})$ have the same distribution. Applying this fact to function $f(Y_{\pi_1},...,Y_{\pi_N})=(Y_{\pi_1},\sum_{k=1}^NY_k)$ implies $(Y_i,\sum_{k=1}^NY_k)$ and $(Y_j,\sum_{k=1}^NY_k)$ share the same joint distribution.

By definition, the conditional expectation $\mathbb{E}[X_i|\sum_{k=1}^NY_k]$ is given by $$ \mathbb{E}[X_iZ]=\mathbb{E}\left[Z \mathbb{E}[X_i|\sum_{k=1}^NY_k]\right],\ Z\in\mathbf{M}\left(\sum_{k=1}^NY_k\right) $$ where $\mathbf{M}\left(\sum_{k=1}^NY_k\right)$ is the closed subspace of $L^2$ consists of all Borel function of $\sum_{k=1}^NY_k$. Multiplying both sides of the above equation by $\alpha_i$, we can also write $$ \mathbb{E}[Y_iZ]=\mathbb{E}\left[Z \mathbb{E}[Y_i|\sum_{k=1}^NY_k]\right]. $$ Then we have $$ \mathbb{E}[Y_iZ]=\mathbb{E}[Y_jZ]=\mathbb{E}\left[Z \mathbb{E}[Y_j|\sum_{k=1}^NY_k]\right] \Longrightarrow \mathbb{E}[Y_i|\sum_{k=1}^NY_k] = \mathbb{E}[Y_j|\sum_{k=1}^NY_k], $$ since $(Y_i,\sum_{k=1}^NY_k)$ and $(Y_j,\sum_{k=1}^NY_k)$ share the same joint distribution and $Z$ is a function of $\sum_{k=1}^NY_k$. Thus, $$ \alpha_i\mathbb{E}[X_i|\sum_{k=1}^NY_k]=\alpha_j\mathbb{E}[X_j|\sum_{k=1}^NY_k]. $$ Summing up both sides with respect to $j$ gives $$ N\alpha_i\mathbb{E}[X_i|\sum_{k=1}^NY_k]=\mathbb{E}[\sum_{j=1}^N\alpha_jX_i|\sum_{k=1}^NY_k]=\sum_{j=1}^N\alpha_jX_i. $$ Therefore, we have $$ \mathbb{E}[X_i|\sum_{k=1}^NY_k]=\frac{1}{N\alpha_i}\sum_{j=1}^N\alpha_jX_i,\ \alpha_i\ne0. $$ For $\alpha_i=0$, it follows from the independence that $\mathbb{E}[X_i|\sum_{k=1}^N\alpha_kX_k]=\mathbb{E}[X_i|\sum_{k\ne i}\alpha_kX_k]=0$.

Implications:

  1. If $\alpha_1=...=\alpha_N=\alpha$, then $\mathbb{E}[X_i|\sum_{k=1}^N\alpha_kX_k]=\mathbb{E}[X_j|\sum_{k=1}^N\alpha_kX_k]=\frac{1}{N\alpha}\sum_{k=1}^NX_k,\ \forall i,j=1,...,N$;
  2. In general, for $\alpha_i\ne \alpha_j$ with $\alpha_i\alpha_j\ne 0$, $\mathbb{E}[X_i|\sum_{k=1}^N\alpha_kX_k]\ne\mathbb{E}[X_j|\sum_{k=1}^N\alpha_kX_k]$, although $X_i$ and $X_j$ are i.i.d.

Does it make sense? Any hint is appreciated!

EDIT 1: Thank @Leander Tilsted Kristensen for pointing out the mistake in the comment. Now I know $(Y_{\pi_1},...,Y_{\pi_N})$ and $(Y_{\pi_1'},...,Y_{\pi_N'})$ do not necessarily have the same distribution. Then, if $\alpha_i$s are not identical, can I make a statement that $\mathbb{E}[X_i|\sum_{k=1}^N\alpha_kX_k]\ne\mathbb{E}[X_j|\sum_{k=1}^N\alpha_kX_k]$ in general?

EDIT 2: Thank @Paresseux Nguyen for the examples and comments. It turns out that in the EDIT 1, "not equal" is still a strong conclusion and we should say the two conditional expectation are "not necessarily" equal.

$\endgroup$
3
  • 2
    $\begingroup$ The claim that $(Y_1, \dots ,Y_N)\sim (Y_{\pi_1} , \dots ,Y_{\pi_N})$ for any permutation $\pi$ is not true. You are allowed to permute the $X$ values, since they are i.i.d., but you are not allowed to permute the constants $\alpha_1,\dots , \alpha_N$. $\endgroup$ Nov 6, 2020 at 16:55
  • $\begingroup$ @Leander Tilsted Kristensen Thanks for the comment. Is it because that $Y_1,...,Y_N$ are independent, but not idendical distributed? $\endgroup$
    – HXW
    Nov 7, 2020 at 0:17
  • $\begingroup$ Yes, that is exactly why. $\endgroup$ Nov 7, 2020 at 11:29

1 Answer 1

1
$\begingroup$

As showed by Kristensen, you kicked off your approach with a not true observation.
In fact, I don't think you can do anything for it in general. I think I can dive a little bit in analysis of this problem to show you where things get complicated.
But I guess it'll be better if I give some examples.
Case 1
Let's say $X=\alpha_1 X_1+\alpha_2X_2+...+\alpha_n X_n$
If $\alpha_1=\alpha_2=...=\alpha_n=1$,
$\mathbb{E}( X_1|X)= \frac{X}{n}$
Case 2
If $(X_k, k \in [n])$ are all standard normal distribution, then
$\mathbb{E}( X_1|X)= \frac{\alpha_1X}{\sum_{k=1}^n \alpha_k^2}$
Case 3
If $ (X_k, k \in [n])$ are all Bernoulli variable $\mathcal{B}(p) (0<p<1)$, $a_k=2^{-k}$, then:
$\mathbb{E}( X_1|X)= [2X]$
$\mathbb{E}( X_2|X)= [4X]-2[2X] $
etc.
**Disclaimer **: I may have mistaken somewhere but the generality stays the same.

$\endgroup$
5
  • $\begingroup$ I see. Thanks! These are really helpful! $\endgroup$
    – HXW
    Nov 7, 2020 at 0:26
  • 1
    $\begingroup$ Zut! I accidentally delete my comment. Anyway, for your updated question, you also cannot have that a.s. Look at my third case, when $X=0$, but you can show they are different variable by calculate the expectation of $X( \mathbb{E}(X_1|X) - \mathbb{E}(X_2|X) )$ (assuming $E(X)=0)$. $\endgroup$ Nov 7, 2020 at 0:36
  • $\begingroup$ Got it. Then should I say that they are "not necessarily" equal? $\endgroup$
    – HXW
    Nov 7, 2020 at 0:42
  • $\begingroup$ Yeah, I think we can say that $\endgroup$ Nov 7, 2020 at 0:44
  • $\begingroup$ Ok. Thanks again! $\endgroup$
    – HXW
    Nov 7, 2020 at 0:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .