Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to use Newton's method to solve the following system of equations, where f and g are functions of x and y. (h,a,f,c,d,b and k are just constants).

$f(y,x)=\left[\begin{array}{c} y^{1}\\ y^{2}\\ y^{3}\\ y^{4} \end{array}\right]-\left[\begin{array}{c} y_{0}^{1}\\ y_{0}^{2}\\ y_{0}^{3}\\ y_{0}^{4} \end{array}\right]+h\left[\begin{array}{cccc} a_{1} & f_{1} & 0 & 0\\ c_{1} & a_{1} & f_{1} & 0\\ 0 & c_{1} & a_{1} & f_{1}\\ 0 & 0 & c_{1} & a_{1} \end{array}\right]\left[\begin{array}{c} y^{1}\\ y^{2}\\ y^{3}\\ y^{4} \end{array}\right]+\left[\begin{array}{cccc} b_{1} & 0 & 0 & 0\\ d_{1} & b_{1} & 0 & 0\\ 0 & d_{1} & b_{1} & 0\\ 0 & 0 & d_{1} & b_{1} \end{array}\right]\left[\begin{array}{c} x^{1}\\ x^{2}\\ x^{3}\\ x^{4} \end{array}\right]-\left[\begin{array}{c} k_{1}\\ 0\\ 0\\ k_{2} \end{array}\right]=0$

$g(y,x)=\left[\begin{array}{c} x^{1}\\ x^{2}\\ x^{3}\\ x^{4} \end{array}\right]-\left[\begin{array}{c} x_{0}^{1}\\ x_{0}^{2}\\ x_{0}^{3}\\ x_{0}^{4} \end{array}\right]+h\left[\begin{array}{cccc} a_{2} & f_{2} & 0 & 0\\ c_{2} & a_{2} & f_{2} & 0\\ 0 & c_{2} & a_{2} & f_{2}\\ 0 & 0 & c_{2} & a_{2} \end{array}\right]\left[\begin{array}{c} y^{1}\\ y^{2}\\ y^{3}\\ y^{4} \end{array}\right]+\left[\begin{array}{cccc} b_{2} & 0 & 0 & 0\\ d_{2} & b_{2} & 0 & 0\\ 0 & d_{2} & b_{2} & 0\\ 0 & 0 & d_{2} & b_{2} \end{array}\right]\left[\begin{array}{c} x^{1}\\ x^{2}\\ x^{3}\\ x^{4} \end{array}\right]-\left[\begin{array}{c} k_{1}\\ 0\\ 0\\ k_{2} \end{array}\right]=0$

Am I right in saying that the Newton Iteration equations be:

$y^{k+1}=y^{k}-\frac{f(y,x)}{\frac{\partial f(y,x)}{\partial y}}$ and $x^{k+1}=x^{k}-\frac{g(y,x)}{\frac{\partial g(x,y)}{\partial x}}$.

Or would the denominators be partial derivatives: $\left(x\frac{\partial f}{\partial y}+y\frac{\partial f}{\partial x}\right)$.

I know it was not necessary to write the entire functions but I have some more questions about the equations depending on the answer to this one.

Many Thanks!

share|cite|improve this question
I've just been thinking a lot about this and realized that I'm not even sure how I can apply Newton Raphson method to these equations. If applying it to the first equations will give x or y? ... Please help if have any ideas. – Hooman Jan 19 '12 at 13:29
Note: It's very possible I'm horribly wrong. With that said: I've never seen Newton's method applied to functions that aren't single variable. Mathworld does not seem to recognize this approach, either. Sorry if I am wrong, but I think Newton's method does not apply to functions that are more than one variable. – 000 Jan 21 '12 at 5:29
There's a method called the Multivariate Newton Raphson Method (MNRM) which is used for functions of more than one variable. But my equations are a bit more complicated. So I'm not quite sure how this could be applied in this case. – Hooman Jan 22 '12 at 12:26
I think I might know the answer now. Will post as soon as I am sure. – Hooman Jan 22 '12 at 14:31

Your equations amount to a system of $8$ scalar equations with $8$ scalar unknowns. Writing $x$ and $y$ as separate vectors is not helping clarity here. Stack them into one $8$-dimensional vector $z$, and similarly stack $f$ and $g$ into one function $h$. Formally speaking, the equation $h(z)=0$ could be solved by the Newton-Raphson method as $$ z_{n+1} = z_n - (Dh(z_n))^{-1} h(z_n) $$
... except this is pointless/tautological when the system is linear. Whatever the starting point is, you get the solution after one step of iteration. Indeed, the Newton-Raphson method amounts to approaching the root by replacing the given system with its linearization around a previously found point, and solving the linear system to find the next approximation. Your system is linear to begin with.

share|cite|improve this answer

What you have is a matrix with two unknowns and four equations. This implies that there isn't necessarily a consistent solution. However, if you are trying to find the nearby local optima solution, you can use a computational method called gradient descent. It is a multivariable analog to Newton's Method. It works a finite (sometimes adaptive, depending on algorithm) step size.

The recurrent equation is: $$\vec x_{i+1}=\vec x_i - h*\vec \nabla f(\vec x)$$ $h$ is the step size, and is usually small. There are advanced techniques for computing the "optimum" step size in a given situation, but the techniques are often not worth the added calculation complexity. As the system gets close to a solution, the gradient often becomes unstable due to numerical conditioning. As a result, it tends to bounce around the actual solution in random directions. However, at this point, the solution found is usually close enough for practical purposes.

share|cite|improve this answer
Since $x,y,f,g$ have four components each, there are $8$ scalar unknowns and $8$ scalar equations. – ᴡᴏʀᴅs Jan 25 '15 at 0:58

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.