1
$\begingroup$

Consider the following cost function. $$J(x)=\sum_{i=1}^n [(x\,a_i+(1-x)\,b_i)-d_i]^2$$ How can I find the first and second derivatives of $J$, with respect to $x$? The variables $b_i$ are dependent on $x$, but $a_i$ and $d_i$ are independent of $x$, where $1\leqslant i \leqslant n$ and $0\leqslant x \leqslant 1$

$\endgroup$
1
$\begingroup$

Yes, we have to apply the chain rule for the first derivative:

$J(x)=\sum\limits_{i=1}^n ((x\cdot a_i+(1-x)\cdot b_i)-d_i)^2$

$J^{'}(x)=\sum\limits_{i=1}^n 2\cdot( (x\cdot a_i+(1-x)\cdot b_i)-d_i)\cdot (a_i-b_i)$

And the second derivative is

$J^{''}(x)=\sum\limits_{i=1}^n 2\cdot(a_i -b_i)\cdot (a_i-b_i)=2\cdot\sum\limits_{i=1}^n (a_i-b_i)^2$

$\endgroup$
  • $\begingroup$ Thanks callculus. I was starting the chain rule, but didn't seem to get to it. Again, thanks. $\endgroup$ – Steven31415 Jun 11 '19 at 12:17
  • $\begingroup$ You´re welcome. $\endgroup$ – callculus Jun 11 '19 at 12:18
  • 1
    $\begingroup$ I missed the first x. I´ve edited my answer. $\endgroup$ – callculus Jun 11 '19 at 12:23
  • $\begingroup$ what if x is a number between zero and 1? $\endgroup$ – Steven31415 Jun 11 '19 at 14:31
  • $\begingroup$ @FromTheoryToPractice If $x=0$, then the first derivative becomes $2\cdot \sum\limits_{i=1}^{n} (b_i-d_i)\cdot (a_i-b_i)$. If $x=1$, then the first derivative becomes $2\cdot \sum\limits_{i=1}^{n} (a_i-d_i)\cdot (a_i-b_i)$. The second derivative is not affected a by $x$. $\endgroup$ – callculus Jun 11 '19 at 14:54
1
$\begingroup$

Assuming that n is not being sent off towards infinity, then we can differentiate the sum. Since the sum is finite, we can expand the sum:

\begin{align*} \sum_{i = 1}^n ((xa_i + (1 - x)b_i) - d_i)^2 = (a_1x + b_1(1 - x) - d_1)^2 + \cdots + (a_nx + b_n(1 - x) - d_n)^2. \end{align*}

Then if we differentiate we have the following:

\begin{align*} \frac{d}{dx}\sum_{i = 1}^n ((xa_i + (1 - x)b_i) - d_i)^2 &= \frac{d}{dx}((a_1x + b_1(1 - x) - d_1)^2 + \cdots + (a_nx + b_n(1 - x) - d_n)^2)\\ &= 2(a_1x + b_1(1 - x) - d_1)(a_1 - b_1) + \cdots + 2(a_nx + b_n(1 - x) - d_n)(a_n - b_n)\\ &= \sum_{i = 1}^n 2(a_ix + b_i(1 - x) - d_i)(a_i - b_i). \end{align*}

Now by differentiating in the same manner, again we get \begin{align*} \frac{d}{dx}\sum_{i = 1}^n 2(a_ix + b_i(1 - x) - d_i)(a_i - b_i) &= \frac{d}{dx}(2(a_1x + b_1(1 - x) - d_1)(a_1 - b_1) + \cdots + 2(a_nx + b_n(1 - x) - d_n)(a_n - b_n))\\ &= 2(a_1 - b_1)(a_1 - b_1) + \cdots + 2(a_n - b_n)(a_n - b_n)\\ &= \sum_{i = 1}^n 2(a_i - b_i)^2. \end{align*}

If n is allowed to head off to infinity, this procedure may not necessarily hold.

$\endgroup$
  • $\begingroup$ K. Mor. n is finite, so the procedure is very helpful. thanks $\endgroup$ – Steven31415 Jun 11 '19 at 12:20
  • 1
    $\begingroup$ You are welcome. I was assuming it was, since it was a cost function. Keep up the good work! $\endgroup$ – K.Mor Jun 11 '19 at 12:21
  • $\begingroup$ and what if x is a number between zero and 1? Does it change ? $\endgroup$ – Steven31415 Jun 11 '19 at 14:32
  • 1
    $\begingroup$ @FromTheoryToPractice no, it should not change. The sum will still be a finite sum. Since $0 < x < 1$ you can then talk about the constraints of your initial function and the constraints on the derivative. That is you can bound the function below by plugging in zero and bound it above by plugging in 1. $\endgroup$ – K.Mor Jun 12 '19 at 0:57
  • $\begingroup$ thanks @K.Mor!! $\endgroup$ – Steven31415 Jun 12 '19 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.