deriving Newton's method for optimization I thought I understood the derivation of Newton's method for finding a minimum,
but just realized I was not being at all careful!
Here are three alternate "derivations".  I think the first two are wrong.
Can anyone confirm my understanding here?
First write the Taylor series
$$
   f(x+\delta) = f(x) + f'(x) \delta  + \frac{1}{2} f''(x) \delta^2 + \ldots
$$
On the right hand side, call the three terms as CT, LT, QT (constant term, linear term, quadratic term).
In each of the 3 derivations below we set the derivative of the left hand side to zero,
then solve for $\delta$, resulting in $\delta = -f'/f''.$
The first "derivation" involves only taking the derivative of only CT, LT. 
\begin{align*}
\frac{d}{d\delta}\left( f(x+\delta) \right)  & 
   = \frac{d}{d\delta}\left( f(x) \right) + \frac{d}{d\delta}\left( f'(x)\delta  \right)
\\ &= f'(x) + f''(x) \delta + \cdots
\\
\end{align*}
Here the linear term in the final result comes from applying the derivative to CT, and the quadratic term in the final result comes from applying the derivative to the LT.
However this derivation requires that
$$
   \frac{d}{d\delta}\left( f(x) \right) = f'(x)
$$
whereas I think it should be that $\frac{d}{d\delta}\left( f(x) \right) = 0$.
The second "derivation" involves applying the Leibnitz(?) product rule to the LT:
\begin{align*}
\frac{d}{d\delta}\left( f(x+\delta) \right)  & = 0 + \frac{d}{d\delta}( f'(x)\delta )
\\
& = \frac{d}{d\delta}( f'(x) ) \cdot \delta   +   f'(x) \cdot \frac{d}{d\delta}( \delta )
\\
& = f''(x) \delta + f'(x) \cdot 1
\\
\end{align*}
I think the flaw here is (again) that  $\frac{d}{d\delta}\left( f'(x) \right) = 0 \ne f''(x)$.
The third "derivation" makes use of LT and QT, while the derivative of the CT is zero:
\begin{align*}
\frac{d}{d\delta}\left( f(x+\delta) \right)  
    & = 0 + \frac{d}{d\delta}( f'(x)\delta )  +  \frac{d}{d\delta}( \frac{1}{2} f''(x)\delta^2 )
\\
   & = 0  \quad+\quad ( 0 + f'(x) )   \quad+\quad   f''(x)\delta
\\
   & = f'(x) + f''(x) \delta
\end{align*}
In the second step, the terms in parentheses are from the product rule,
and I believe $\frac{d}{d\delta} f'(x) = 0$.
 A: You are asking for a derivation of Newton's method to find a minimum. The procedure with which I am familiar (and to which I think you refer in your answer) involves using Newton's root finding method on $f'(x)$ to find an extremum of $f(x)$. If you don't need a proof of Newton's method for root finding, then all you need to do in order to derive the optimization method is to check that the conditions for applying the root finding method to $f'(x)$ are satisfied, and then confirm that the root $\alpha$ of $f'$ you get out of the root finding method is indeed a local minimum of $f$ (as opposed to a maximum or a saddle point). You'll need to do a little more work if you want to make sure it's a global minimum.
If you want to prove the convergence of Newton's method for root finding, which I think you might want to do in light of the series expansions you take, I recommend the Wikipedia entry for Newton's Method. This entry also includes a discussion of the conditions under which Newton's method is guaranteed to converge.
A: Ok, I think I have an answer.
I was studying optimization, where you typically set to zero the derivative of the variable that you are trying to solve for.  In the Newton case above, that variable is $\delta$.
However, the minimum of the curve is at the location where the derivative with respect to $x$ is zero, so that is what should be used, even though $\delta$ is what is being solved for.
