Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Let $f: \mathbb{R}^n \longrightarrow \mathbb{R}$ and let $g: \mathbb{R} \longrightarrow \mathbb{R}$ given by $g(\theta):= f(\theta {\bf y} + (1 − \theta){\bf x})$.

I want to calculate the derivative $g'$.

I set $z=\theta {\bf y}+ (1 − \theta){\bf x}$, then:

$$ \frac{\mathrm{d}g}{\mathrm{d}\theta}= \frac{\mathrm{d}f}{\mathrm{d}\theta}= \frac{\mathrm{d}f}{\mathrm{d}z} \frac{\mathrm{d}z}{\mathrm{d}\theta}= \nabla f^\mathsf{T} ({\bf y}-{\bf x}) $$

Is this formally correct? If it is not wrong, is it possible to write it better?

share|improve this question
    
What does the $ {}^\mathsf{T}$ signify? –  Pedro Tamaroff May 2 '13 at 23:16
    
@PeterTamaroff: Transposition as the gradient is a vector. I am in doubt probably $\frac{\mathrm{d}f}{\mathrm{d}z}$ should be transposed too. –  antonio May 3 '13 at 7:41
add comment

3 Answers

There's a way to make this a little better. First of all, note that if we define $\alpha: \mathbb{R} \to \mathbb{R}^n$ by $\alpha(\theta)=\theta y+(1-\theta)x$ then $\alpha$ is a differentiable function of $\theta$ at every point of $\mathbb{R}$. Now, it's pretty clear that $g = f\circ \alpha$, because we have $g(\theta)=f(\alpha(\theta))$ which is what you want.

Now we simply apply the chain rule in it's general form (if you never saw this way of writing the chain rule, look at Spivak's Calculus on Manifolds):

$$Dg(\theta)=Df(\alpha(\theta))\circ D\alpha(\theta)$$

Now, in terms of matrices we have the following:

$$g'(\theta)=f'(\alpha(\theta))\alpha'(\theta)$$

However we have $\alpha'(\theta)=y-x$ so that understading $y-x$ as it's corresponding column matrix we have:

$$g'(\theta)=f'(\alpha(\theta))(y-x)$$

Now, since $f$ maps $\mathbb{R}^n \to \mathbb{R}$, we have $f'(\alpha(\theta))$ a row matrix and the expression above is the multiplication of the row matrix by the column matrix. Thinking of linear transformations, we are applying the transformation $Df(\alpha(\theta))$ into the vector $y-x$ so that we have:

$$g'(\theta) = Df(\alpha(\theta))(y-x)$$

Finally, if you want to write using gradients, remember that if $f$ is a scalar fields, applying it's derivative into some vector equals making the inner product between the gradient and the vector, so tht we obtain your expression:

$$g'(\theta) = \nabla f(\alpha(\theta))\cdot(y-x)$$

And just to finish a little geometrical interpretation: $\alpha$ is a curve, indeed $\alpha$ is a line. When you compute $f\circ \alpha$ you get the values of $f$ just along $\alpha$ (you restric $f$) and then $g'(\theta)$ is the derivative of the function along the line, so you are getting the directional derivative of $f$ in the direction of the line.

share|improve this answer
add comment

Take ${\bf x}=(x_1,x_2,\dots,x_n),{\bf y}=(y_1,y_2,\dots,y_n)$. Then you have $$g(\theta)=f(\theta y_1+(1-\theta)x_1,\theta y_2+(1-\theta)x_2,\dots,\theta y_n+(1-\theta)x_n)$$

Note that you have a something of the form $$g(\theta)=f(H_1(\theta),H_2(\theta),\dots,H_n(\theta))$$

All you have to do is apply the chain rule.

In particular, if we have $F:\Bbb R\to \Bbb R^n$ with $$F(\theta)=(H_1(\theta),H_2(\theta),\dots,H_n(\theta))$$

then $$F'(\theta)=(H_1'(\theta),H_2'(\theta),\dots,H_n'(\theta))$$

and $$ (f\circ F)'(\theta)=\nabla f( F(\theta))\cdot F'(\theta)$$

Now, as you correctly noted $$H_n(\theta)=\theta y_n+(1-\theta)x_n$$ is such that $$H_n'(\theta)=y_n-x_n$$ so that $$F'(\theta)={\bf y-x}$$

So you ultimately obtain $$g'(\theta)=\nabla{\bf f}(\theta{\bf y}+(1-\theta){\bf x})\cdot ({\bf x-y})=\nabla{\bf f}({\bf z})\cdot ({\bf x-y})$$ where ${\bf z}=\theta{\bf y}+(1-\theta){\bf x}$

share|improve this answer
add comment

Yes what you are doing is correct. Perhaps if you want to add more steps, note that

$$ z_{i}=\theta y_{i}+\left(1-\theta\right)x_{i} $$

so that

$$ \frac{\partial z_{i}}{\partial\theta}=y_{i}-x_{i}. $$

Then,

$$ \frac{dg}{d\theta}=\frac{\partial f}{\partial z_{1}}\frac{\partial z_{1}}{\partial\theta}+\cdots+\frac{\partial f}{\partial z_{n}}\frac{\partial z_{n}}{\partial\theta}=\frac{\partial f}{\partial z_{1}}\left(y_{1}-x_{1}\right)+\cdots+\frac{\partial f}{\partial z_{n}}\left(y_{n}-x_{n}\right), $$

which is the inner product you had above. Note that $\nabla f$ is evaluated at $z$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.