Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am learning about Bayesian statistics and I'm currently doing loss functions. Let $f(\theta | \mathbf{x} ) $ be a posterior pdf . Let $F(\theta | \mathbf{x} ) $ be the associated distribution function. I want to differentiate $F(a - D| \mathbf{x} )$ with respect to $D$. In this case $D$ is the "decision" which is to be optimised and $a$ is constant. I am having a problem here: $$\frac{d}{d D}F(a - D| \mathbf{x} ) = \frac{d}{d D} \left( \int_{-\infty}^{a - D} f(\theta|\mathbf{x}) d \theta \right)$$ $$=f(a - D | \mathbf{x})$$

Is this correct ? I think it might be wrong. Should it be $=-f(a - D | \mathbf{x})$ because I have to apply the chain rule somewhere ?? Or is something else wrong ? I know there are some issues about integration under differentiation but my teacher said I don't need to worry about that now, and just use this: $$\frac{d}{dy} \int_{-\infty}^y f(t) dt=f(y)$$

I'm doing self-study (with a bit of teacher guidance in his own time so I don't like to ask him too much) but I feel a bit out of depth now and school breaks up for the holidays next week !

share|improve this question

migrated from stats.stackexchange.com Jul 13 '12 at 20:01

This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.

    
Yes, it should be $=-f(a-D|x)$. Intuitively, and writing loosely, increasing $D$ decreases the upper limit of integration, which decreases the integral if $f(a-D|x) > 0$. –  jbowman Jul 13 '12 at 20:39
    
@jbowman : That is what I suspected, I do follow the intuition, but how to make it less "loose" ? I had the notion that I have to apply the chain rule and therefore multiply by $\frac{d}{dD}(a-D)=-1$ somewhere, but I don't know where I should put that..... –  Joe King Jul 13 '12 at 21:03

2 Answers 2

up vote 3 down vote accepted

Think of this as a special case of differentiating $h(f(D))$ with respect to D:

$h(f(D)) = \int_{-\infty}^{f(D)}g(x)\text{d}x$

where $\text{d}h/\text{d}D = h'(f(D))f'(D)$ via the chain rule, as you thought. In your case, $f(D) = a-D$, so $h'(f(D)) = g(a-D)$, as you know, and $f'(D) = -1$.

share|improve this answer
    
Thanks, so is it correct to write this: $$\frac{d}{d D}F(a - D| \mathbf{x} ) = \frac{d}{d D} \left( \int_{-\infty}^{a - D} f(\theta|\mathbf{x}) d \theta \right)$$ $$=f(a - D | \mathbf{x}) \times \frac{d}{d D}(a - D)$$ $$=-f(a - D | \mathbf{x})$$ ? –  Joe King Jul 14 '12 at 7:10
    
What's confusing me is the application of the chain rule together with the rule for integration under differentiation......something just doesn't seem right with what I wrote in the comment above. –  Joe King Jul 14 '12 at 7:21
    
You have it correctly in your comment. The rule for differentiating an integral w.r.t. one of the limits of integration tells you how to find the derivative with respect to the limit, which is just a special case of a general function $f(x)$ (i.e., f(x) = $\int_a^x g(x)\text{d}x$.) When $x$ is itself a function $x(z)$, and you really want the derivative w.r.t. $z$, the rule only gives you $\text{d}f/\text{d}x$, and you do still need the chain rule to get all the way to $\text{d}f/\text{d}z$. –  jbowman Jul 14 '12 at 13:50

I think the key here is to abstract away the irrelevant information. That might be easier said than done, but consider this line of reasoning:

  • The question is about a derivative.

  • It's not about numerical calculation of the derivative; symbolically, we know that $dF = f$. (Observe how the dependence on $\mathbf{x}$ is dropped in this simplified notation: because $\mathbf{x}$ is fixed throughout the problem, it likely plays no essential role.)

  • The difficulty appears to stem from differentiating with respect to $D$ when the argument of $F$ is supplied as $a-D$ instead of $\theta$.

  • Although there is a context or "story line" in which decisions, losses, and a Bayesian procedure are involved, this context appears to offer no additional mathematical information.

A propos the last point, one could hope that a solution might be motivated by the context, but ultimately a rigorous mathematical answer should not need to invoke any element of this context. This makes it a good candidate to abstract away.

Whence, let's try reframing the question:

Given that the derivative of $F$ is $f$, to produce an expression for $\frac{d}{dD}F(a-D)$.

Many people visualize this situation as a composition of functions, written thus:

$$D \to a - D \to F(a-D)$$

The first function sends $D$ to $a-D$; the second is $F$.

The Chain Rule states that when everything makes sense, the composition has a derivative and the derivative is the product of the derivatives of the arrows. The derivative of $D \to a-D$ is $-1$ (an elementary calculation) while the derivative of $F$ is (symbolically) $f$, applied (as in the diagram) to $a-D$, whence the derivative of this composition is simply $-1 \times f(a-D)$. Reverting to the original notation, we would write

$$\frac{d}{dD} F(a-D|\mathbf{x}) = -f(a-D|\mathbf{x}).$$


Please note that the potentially confusing question of differentiating a limit of an integral doesn't even come up: that's one advantage of the clarity and focused afforded by this method of abstracting out the inessential parts of a "real world" problem.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.