Differentiating a specific cost function Consider the following cost function.
$$J(x)=\sum_{i=1}^n [(x\,a_i+(1-x)\,b_i)-d_i]^2$$
How can I find the first and second derivatives of $J$, with respect to $x$? The variables $b_i$ are dependent on $x$, but $a_i$ and $d_i$ are independent of $x$, where $1\leqslant i \leqslant n$ and $0\leqslant x \leqslant 1$
 A: Yes, we have to apply the chain rule for the first derivative:
$J(x)=\sum\limits_{i=1}^n ((x\cdot a_i+(1-x)\cdot b_i)-d_i)^2$
$J^{'}(x)=\sum\limits_{i=1}^n 2\cdot( (x\cdot a_i+(1-x)\cdot b_i)-d_i)\cdot (a_i-b_i)$
And the second derivative is
$J^{''}(x)=\sum\limits_{i=1}^n 2\cdot(a_i -b_i)\cdot (a_i-b_i)=2\cdot\sum\limits_{i=1}^n  (a_i-b_i)^2$
A: Assuming that n is not being sent off towards infinity, then we can differentiate the sum. Since the sum is finite, we can expand the sum:
\begin{align*}
\sum_{i = 1}^n ((xa_i + (1 - x)b_i) - d_i)^2 = (a_1x + b_1(1 - x) - d_1)^2 + \cdots + (a_nx + b_n(1 - x) - d_n)^2.
\end{align*}
Then if we differentiate we have the following:
\begin{align*}
\frac{d}{dx}\sum_{i = 1}^n ((xa_i + (1 - x)b_i) - d_i)^2 &= \frac{d}{dx}((a_1x + b_1(1 - x) - d_1)^2 + \cdots + (a_nx + b_n(1 - x) - d_n)^2)\\
&= 2(a_1x + b_1(1 - x) - d_1)(a_1 - b_1) + \cdots + 2(a_nx + b_n(1 - x) - d_n)(a_n - b_n)\\
&= \sum_{i = 1}^n 2(a_ix + b_i(1 - x) - d_i)(a_i - b_i).
\end{align*}
Now by differentiating in the same manner, again we get
\begin{align*}
\frac{d}{dx}\sum_{i = 1}^n 2(a_ix + b_i(1 - x) - d_i)(a_i - b_i) &= \frac{d}{dx}(2(a_1x + b_1(1 - x) - d_1)(a_1 - b_1) + \cdots + 2(a_nx + b_n(1 - x) - d_n)(a_n - b_n))\\
&= 2(a_1 - b_1)(a_1 - b_1) + \cdots + 2(a_n - b_n)(a_n - b_n)\\
&= \sum_{i = 1}^n 2(a_i - b_i)^2.
\end{align*}
If n is allowed to head off to infinity, this procedure may not necessarily hold.
