Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm currently looking at this example problem from my course notes (introductory numerical analysis):

enter image description here

and am a bit confused about how to write the equation for Newton's method. From its general form, I would conjecture that it is simply

$$ w_{i+1} = w_i - \frac{f(t_i, w_i)}{f_y(t_i, w_i)} $$

but that seems incorrect as $f$ is a function of multiple variables. Is this correct or should I be considering changes in the variable $t$ as well?

share|cite|improve this question

To get the multi-variable analogue, simply replace $\frac{1}{f'(x)}$ with the inverse Jacobian of the system $J^{-1}$, so \begin{equation} \textbf{x}_{i+1} = \textbf{x}_i - J^{-1}\mathbf{f}(\mathbf{x}_i), \end{equation} where the entries of $J$ \begin{equation} j_{ik} = \frac{\partial f_i}{\partial x_k} \end{equation} are all the first partial derivatives of all functions.

Note there are computationally efficient ways of computing the $J^{-1}\mathbf{f}(\mathbf{x}_i)$ which don't involve calculating the Jacobian, though this is beyond the scope of this question.

share|cite|improve this answer

When you write Newton formula as given in your book, what you basically write is that $$f(x)=f(x_0)+(x-x_0)f'(x_0)=0$$ from which $$x-x_0=-\frac{f(x_0)}{f'(x_0)}$$ To stay simple, let us consider now two equations $f(x,y)=0$ and $g(x,y)=0$. We can now write $$f(x,y)=f(x_0,y_0)+(x-x_0)f'_x(x_0,y_0)+(y-y_0)f'_y(x_0,y_0)=0$$ $$g(x,y)=g(x_0,y_0)+(x-x_0)g'_x(x_0,y_0)+(y-y_0)g'_y(x_0,y_0)=0$$ from which $(x-x_0)$ and $(y-y_0)$ can be calculated solving the two linear equations for the two unknowns. As you see, it is very similar but, in the first case, we use the derivative of the equation while, in the second case, we use something slightly more complex (called the Jacobian of the system) which is build from the derivatives of each equation with respect to ech variable.

I shall not enter here in more details. Let me know if this make things clearer to you.

share|cite|improve this answer

Staying in your notation, it should be $$ w_{i+1}^{m+1} = w_{i+1}^{m} - F_y(w_{i+1}^{m})^{-1}F(w_{i+1}^{m}) $$ with $F(y)=y-w_i-hf(t+h,y)$, so that $F_y(y)=I-hf_y(t+h,y)$.

Using $w_{i+1}^{0}=w_i$ as a crude predictor, the first approximation is \begin{align} w_{i+1}^{1} &=w_i+h(I-hf_y(t+h,w_i))^{-1}f(t_{i+1},w_i) \\ &=w_i+hf(t_{i+1},w_i)+h^2(I-hf_y(t+h,w_i))^{-1}f_y(t+h,w_i)f(t_{i+1},w_i) \end{align} This is related to the Rosenbrock integrators for stiff systems.

As better predictor you might take the forward Euler step $$ w_{i+1}^{0}=w_i+hf(t_i,w_i) $$

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.