Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

background: I'm working on a problem that uses the implicit function theorem to show the existence of a solution. I have a continuously differentiable function $f(x,y)=0$ with nonzero Jacobian at a point $(a,b)$. A direct corollary of the implicit function theorem asserts that for sufficiently small changes in $a$, say $a+\delta$ there is a $b+\epsilon$ such that $f(a+\delta,b+\epsilon)=0$.

Question: How do one find such a new solution in a concrete problem?

Example: Let's look at a simple function like $f(x,y)=x^2+y^2-1=0$. $(\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2})$ is a solution, and the Jacobian of $f$ is nonzero at this point. How do you find a solution of the form $(\frac{\sqrt{2}}{2}+0.1,y)$?

Note: I know that the above example can be solved directly, but I want to make use of a method (something like Newton's method maybe) for higher dimensional problems, where $x=(x_1,\ldots,x_m)$, and $y=(y_1,\ldots,y_n)$

share|cite|improve this question
For a numerical answer, I would use Newton's method. (Not that it matters, but many proofs of the IFT and Newton method convergence are very similar.) You need to have a scheme to deal with approaches to points where the Jacobian becomes singular. The IFT gives the derivative of the solution at a solution, so this can be used to compute an initial guess for the perturbed solution. – copper.hat Sep 10 '12 at 17:47
up vote 1 down vote accepted

In your example, the IFT guarantees the existence of a $C^1$ function $y$ defined in the neighborhood of $\frac{1}{\sqrt{2}}$ such that $f(x,y(x)) = 0$. Furthermore, $\frac{\partial y(\frac{1}{\sqrt{2}})}{\partial x} = -(\frac{\partial f(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}))}{\partial y})^{-1} \frac{\partial f(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}))}{\partial x} = -1$.

To solve $f(\frac{1}{\sqrt{2}}+0.1, y) = 0$, use Newton's method starting from the initial estimate $y_0 = y(\frac{1}{\sqrt{2}}+0.1) \approx y(\frac{1}{\sqrt{2}})+\frac{\partial y(\frac{1}{\sqrt{2}})}{\partial x} 0.1 = \frac{1}{\sqrt{2}}-0.1$.

This converges quickly (quadratically, of course):

y_1 = 0.590635214224
y_2 = 0.59040553564
y_3 = 0.590405490966
y_4 = 0.590405490966

Here is the Python script that produced the above...

from math import *

# initial values...
x = 1/sqrt(2.0)+0.1
y = 1/sqrt(2.0)-0.1

# update y using Newton's method...
for k in range(1, 5):
    y = y - (pow(x, 2.0)+pow(y, 2.0)-1)/(2.0*y)
    print "y_%s = %s" % (k, y,)
share|cite|improve this answer
I believe in the 4th line you mean $y_0= \dots = \frac{1}{\sqrt{2}}-0.1$. Is that correct? For the next step do you compute the derivative at $y_0$? But that is not a point on the graph any more, doesn't that make a problem? – Keivan Sep 10 '12 at 22:59
@Keivan: Thanks for catching that! – copper.hat Sep 10 '12 at 23:37
@Keivan: No, you compute the derivative at the current point. – copper.hat Sep 10 '12 at 23:38
Well, that is my problem. I am computing the derivative on some point which is not on the curve any more. Maybe I should go and study the multivariable Newton's method again. I think I'm missing something big here. Thanks for the help. – Keivan Sep 11 '12 at 3:06
@Keivan: If the point is on the curve $f(\frac{1}{\sqrt{2}}+0.1,y) = 0$, then you are finished, so until then, you must be 'off the curve' - the whole point of using Newton's method is to 'get back on the curve'. – copper.hat Sep 11 '12 at 4:58

The IFT tells you that you can solve for $y$ as a function of $x$ in a neighborhood of $a$ in such a way that $y(a)=b$. To approximate $y(a+\delta)$ you can use the Taylor expansion of $y$ around $x=a$. For this you need to know the derivatives of $y$ with respect to $x$ at $x=a$, which can be computed by implicit differentiation.

In your example you want $$ y\Bigl(\frac{\sqrt2}{2}+0.1\Bigr)\approx y\Bigl(\frac{\sqrt2}{2}\Bigr)+\frac{dy}{dx}\Bigl(\frac{\sqrt2}{2}\Bigr)\times0.1=\frac{\sqrt2}{2}+\frac{dy}{dx}\Bigl(\frac{\sqrt2}{2}\Bigr)\times0.1 $$ To compute the derivative we differentiate the equation $f(x,y)=0$: $$ 2\,x+2\,y\frac{dy}{dx}=0\implies\frac{dy}{dx}=-\frac{x}{y}\implies\frac{dy}{dx}\Bigl(\frac{\sqrt2}{2}\Bigr)=-1. $$

share|cite|improve this answer
Thanks, but then how do you make the approximation better? By calculating the second, third ... derivatives, or by iteration? If the latter, I'm not sure how to iterate it. – Keivan Sep 10 '12 at 19:54
If you use the Taylor expansion, by computing higher derivatives. Another possibility is to use the Newton-Raphson method. In each iteration you would use implicit differentiation to compute the derivatives. – Julián Aguirre Sep 10 '12 at 20:22
Here is what I don't understand, while using the Newton's method, after the first iteration what shall I use as the initial point in the second iteration? – Keivan Sep 10 '12 at 20:48

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.