Suppose I have a random variable $X$, with support in $\mathbb R$, cdf $F$ and density $f$. Therefore, we have

$P(X\leq \theta)=\int_{-\infty}^\theta f(x)dx=\int_{-\infty}^\theta dF(x)$

Then, by the Leibniz Rule, in the first equality

$\frac{\partial P(X\leq\theta)}{\partial\theta}=f(\theta)$

But in the second equality:

$\int_{-\infty}^\theta dF(x)=\int \mathbb 1(x\leq\theta)dF(x)$, since $\partial\mathbb1(x\leq\theta)/\partial\theta=0$ almost everywhere, then

$\frac{\partial P(X\leq\theta)}{\partial\theta}=0$

  • $\begingroup$ Maybe see this: mathoverflow.net/q/43792 $\endgroup$ – Sean Roberson Aug 13 '17 at 0:18
  • $\begingroup$ This is the origin of the identity $$ \frac{d}{d\theta}1(x\le \theta) = \delta(\theta-x)$$. $\endgroup$ – spaceisdarkgreen Aug 13 '17 at 0:31
  • $\begingroup$ can you elaborate further please? $\endgroup$ – julian.marr Aug 13 '17 at 0:33

First, note that this is not specific to probability or the Stieljes integral. You could have asked the question about a plain vanilla integral $$\int_{-\infty}^\theta f(x)dx = \int_{-\infty}^\infty H(\theta-x)f(x)dx$$ where $H$ is the step function. Actually, we can simplify even further and consider derivative with respect to $\theta$ of $$\theta= \int_0^\theta dx = \int_0^\infty H(\theta-x)dx$$ where $\theta > 0.$ Everything about your question still applies here.

We know the derivative of the LHS is one. So what's wrong with the logic of bring the derivative inside the integral and saying it's zero. First, it's not always okay to bring a derivative under the integral sign. Second, I understand the logic that it's zero almost everywhere so thus the integral must be zero, but that's applying rules about functions to something that's fundamentally a distribution (or at least you must treat it as a distribution if you bring the derivative inside). By the same logic the Dirac delta function is zero almost everywhere so it must integrate to zero... in fact we shall see that this analogy is exact.

Let's go back to the definition of the derivative. We want $$\lim_{h\to0}\frac{f(\theta+h)-f(\theta)}{h} = \lim_{h\to 0} \frac{1}{h}\left(\int_0^\infty H(\theta+h-x)dx - \int_0^\infty H(\theta-x)dx\right).$$ Of course these integrals are just silly ways of writing $\theta+h$ and $\theta$ so they converge and there's no question of us combining them under one and we can write our expression $$ \lim_{h\to 0} \int_0^\infty \frac{H(\theta+h-x)-H(\theta-x)}{h}dx.$$

Now we face the all-important question of whether we can bring the limit inside the integral. If we do, we know that we get zero at almost every $x.$ But at the all-important point $x=\theta$ the derivative is undefined. This is where all the action is.

So instead of trying to bring the limit inside, let's look at what the inside function looks like as a function of $x$ for fixed $h.$ It's a rectangle of width $h$ and height $\frac{1}{h}$ with support between $\theta$ and $\theta+h$. Hmm. This is a rectangle of area one that gets arbitrarily tall and thin as $h\to 0.$ So it is an approximation to a Dirac delta. Of course for any $h\ne 0$ the integral is one so the limit is also one, as we knew it had to be.

So, we see the "derivative of the step function" behaves exactly as a delta function. This gives us the distribution identity $$ \frac{d}{d\theta} H(\theta-x) = \delta(\theta-x)$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.