6
$\begingroup$

I often hear the term "Darth Vader Rule" when calculating the expected value using the survival function and taking the integral where it is defined.

I am not quite sure why it is called that (is it customary?) and I would also like to know a formal proof of it. I tried to look around, but I have a feeling that the name of this rule is not official and I cannot seem to find it right away.

$\endgroup$
  • 11
    $\begingroup$ It's when you question a proof, and the speaker finds your lack of faith to be disturbing. $\endgroup$ – Asaf Karagila Sep 4 '14 at 20:17
  • 1
    $\begingroup$ Before anyone else votes to close this question, please note that web search for "darth vader rule" does show that the term is widely-used in actuarial circles. So presumably the question, which is "what is a proof of it", is perfectly clear to someone who is familiar with actuarial matters. $\endgroup$ – MJD Sep 4 '14 at 20:22
  • $\begingroup$ See for example here. Google first! $\endgroup$ – Winther Sep 4 '14 at 20:22
  • 2
    $\begingroup$ For intuitive explanations as well as more formal proofs, see the answers to this question. $\endgroup$ – Dilip Sarwate Sep 4 '14 at 20:33
  • $\begingroup$ statsravingmad.wordpress.com/2010/01/25/… says the joke is that Darth Vader was a "true survivor". I still don't get it. $\endgroup$ – Nate Eldredge Sep 4 '14 at 21:06
6
$\begingroup$

A basic proof uses Lebesgue Integration.

Let $S(x)$ be a survival function on $x\in [0,\infty]$, then $S(x)$ is a monotonically decreasing function starting at $S(0)=1$ and $\lim\limits_{x\rightarrow \infty} S(x)= 0$.

Now, lets calculate the area under the curve using a Lebesgue Sum, of $S(x)$.

$L_S:= \sum\limits_{\eta_i\in \chi_S} \Delta(\eta_i)\mu(S^{-1}(\eta_i))$

Where:

  • $\chi_S$ is a partition of the range of $S(x)$ into a set of intervals.
  • $\Delta(\eta_i)$ is the length of interval $\eta_i \in \chi_S$
  • $\mu(S^{-1}(\eta_i))$ is the Lebesgue measure $\mu$ (i.e., total length) of the interval on the x-axis where $S(x)\geq \inf \eta_i$

Such an integral can be hard to interpret. However, since $S(x)$ is monotonic-decreasing, we know that the set of $x$ values in each term of the summation will have a special property: $\mu(S^{-1}(\eta_i))=x_i:S(x)=\inf \eta_i$, which means we can dispense with the Lebesgue measure and just use the actual function inverse:

$L_S := \sum\limits_{\eta_i\in \chi_S} \Delta(\eta_i)S^{-1}(\eta_i)$

Now, lets take the limit of the Lebesgue sum to get a Lebesgue Integral:

$\lim\limits_{\Delta(\eta_i)\rightarrow 0} \sum\limits_{\eta_i\in\chi_S} \eta_i\mu(S^{-1}(\eta_i)) = \int_0^1 S^{-1}(z)dz$ [This can be envisioned as the limit of a series of stacked rectangles (i.e. a Riemann sum on the inverse of S)].

However, note that $dz = dS = dP$; thus, an interval on the y-axis represents a probability, and the limit of this interval represents a density, so we can re-write the integral using the fact that $\int f(x) dx = \int f^{-1}(y) dy$:

$\int_0^1 S^{-1}(z)dz = \int_0^{\infty} xdS=\int_0^{\infty} xdP = E[X]\;\;\text{ where } F_X(x)=1-S(x)$

$\endgroup$
  • $\begingroup$ wow, I did not think that it would be so beyond my knowledge. Thank you for your help, though. Hopefully someday I will come back here and be able to understand your proof. $\endgroup$ – hyg17 Sep 5 '14 at 7:27
  • $\begingroup$ @hyg17 sorry, I didn't intend to confuse...was trying to satisfy the "formal" part of your request for a proof. I am actually an applied math person myself, so the way I actually view this is simply as a regular integral, except you treat the Y axis as the domain, and the X axis as the range (i.e., value of the function). This will get you the same result: dy=dP and the horizontal "rectangles" (in a Riemann sense) will give the area under the curve (once you integrate over all values of Y) $\endgroup$ – user76844 Sep 5 '14 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.