Suppose $X_1,X_2,\ldots, X_n$ are i.i.d. random variables. Is there some way to determine the distribution of $(X_1,\ldots,X_n)$ given $S_n := X_1+\ldots+X_n$?

In the discrete case it is easy just using the definition of conditional expected value, but what about in the continuous case?

  • $\begingroup$ You could take the $n$-th root of the characteristic function of $S_n$. $\endgroup$ – Angina Seng May 14 '17 at 4:17
  • 1
    $\begingroup$ @LordSharktheUnknown I don't think that helps. Although the $X_i$ will be identically distributed given $S_n$, they will not be independent. $\endgroup$ – nullUser May 14 '17 at 4:41
  • $\begingroup$ You could define $X=(X_1, ..., X_n)$, $x=(x_1, ..., x_n)$, define the event $A_x = \{X \leq x\}$, and then $$ P[X \leq (x_1, ..., x_n)| Y=y] = \frac{f_{Y|A_x}(y|A_x)P[X_1\leq x_1]...P[X_n\leq x_n]}{f_Y(y)}$$ Then you can take partial derivatives with respect to $x_i$ if you like. I think your $S_n$ morphed into my $Y$, although you could use the above in more general situations when $Y$ is not necessarily the sum. $\endgroup$ – Michael May 14 '17 at 4:46
  • 2
    $\begingroup$ When every $X_i$ has PDF $f$, their sum $$S=\sum_i X_i$$ has PDF $g=f^{\ast n}$ the convolution of $f$ with itself $n$ times, and the distribution of $X=(X_i)_i$ conditional on $S$ has conditional PDF $$f_{X\mid S}(x\mid s)=g(s)^{-1}\prod_i f(x_i)$$ with the notation $x=(x_i)_i$, on the set $$s=\sum_i x_i$$ Since your post also mentions "conditional expected value(s)", let me add that these are much simpler, since, for every $i$, $$E(X_i\mid S)=\frac1nS$$ $\endgroup$ – Did May 14 '17 at 7:26
  • 2
    $\begingroup$ @Did the part that is tripping me up is that the set $s=\sum_i x_i$ has Lebesgue measure zero. Presumably I should be able to calculate $P(X\in A|S=s) = \int_A f_{X|S}(x|s)\,dx$, but this just gives $0$ since the support of $f_{X|S}$ has measure zero. Should I be integrating with respect to a different measure? Also if you know of a reference that handles these subtleties carefully I would appreciate it. $\endgroup$ – nullUser May 14 '17 at 19:43

Expanding on my first comment...

General formula

Let $S$ be a random variable with PDF $f_S(s)$ for $s \in \mathbb{R}$. Let $A$ be an event with $P[A>0]$. Then: $$ P[A|S=s] = \frac{f_{S|A}(s|A)P[A]}{f_S(s)} $$ For intuition about this formula, it is easy to verify that for any interval $[a,b]$: \begin{align} \int_{s=-\infty}^{\infty} P[A|S=s]f_S(s)ds &= P[A] \\ \int_{s \in [a,b]} P[A|S=s] f_S(s) &= P[A \cap \{ S \in [a,b]\}] \end{align}

Application to your problem

Let $\{X_i\}_{i=1}^n$ be i.i.d., let $S=\sum_{i=1}^n X_i$. We want to find $P[(X_1, \ldots, X_n) \leq (x_1, ..., x_n) | S=s]$ for all relevant values of $s \in \mathbb{R}$. For notational simplicity define $X=(X_1, ..., X_n)$, $x=(x_1, ..., x_n)$, $A_x = \{X\leq x\}$. Note that: $$P[A_x] = P[X\leq x] = P[X_1\leq x_1]\cdots P[X_n\leq x_n] $$ Assume $S$ has PDF $f_S(s)$. We want to compute $P[A_x|S=s]$. Applying the above formula gives: \begin{align*} P[A_x | S=s] &= \frac{f_{S|A_x}(s|A_x)P[A_x]}{f_S(s)} \\ &= \frac{f_{S|A_x}(s|A_x)P[X_1\leq x_1]\cdots P[X_n\leq x_n]}{f_S(s)} \end{align*} You can find $f_S(s)$ by $n$-fold convolution of the PDFs of $f_X(s)$ (assuming such exist). You can find $f_{S|A_x}(s|A_x)$ by: \begin{align} f_{S|A_x}(s|A_x) &= \frac{d}{ds} P[S \leq s| A_x] \\ &= \frac{1}{P[A_x]}\frac{d}{ds}P[\{S\leq s \} \cap A_x]\\ &= \frac{\frac{d}{ds} P[X_1\leq x_1, ..., X_n \leq x_n, X_1+...+X_n\leq s]}{P[X_1\leq x_1]\cdots P[X_n\leq x_n]} \end{align}

Computing probabilities with the conditional CDF

Notice that the above gives the conditional cumulative distribution function (CDF) rather than the conditional PDF. Conditional PDFs given $X_1 + ... +X_n=s$ are hard to define since, as you note, this restricts $(X_1, ..., X_n)$ to a multidimensional set of measure zero in $\mathbb{R}^n$. So real-valued functions defined over that measure-zero set would integrate to 0, not to 1. One would need to use PDFs with multidimensional impulses, which are tricky. However, there is no need to use impulses: Working with the conditional CDF gives all you need.

For example, suppose $n=2$ and let $B$ be the line segment in $\mathbb{R}^2$ between the points $(0,1)$ and $(1,0)$. How do we compute $P[(X_1, X_2) \in B | S=1]$? We just observe:

$$P[(X_1, X_2) \in B | S=1] = P[\underbrace{X_1 \leq 1, X_2 \leq 1}_{A_{(1,1)}} | S=1] $$

A 2-d picture to illustrate the above equation would be great, but I do not know how to post a picture on stackexchange.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.