# Approximating a solution where the existence is guaranteed by IFT

background: I'm working on a problem that uses the implicit function theorem to show the existence of a solution. I have a continuously differentiable function $f(x,y)=0$ with nonzero Jacobian at a point $(a,b)$. A direct corollary of the implicit function theorem asserts that for sufficiently small changes in $a$, say $a+\delta$ there is a $b+\epsilon$ such that $f(a+\delta,b+\epsilon)=0$.

Question: How do one find such a new solution in a concrete problem?

Example: Let's look at a simple function like $f(x,y)=x^2+y^2-1=0$. $(\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2})$ is a solution, and the Jacobian of $f$ is nonzero at this point. How do you find a solution of the form $(\frac{\sqrt{2}}{2}+0.1,y)$?

Note: I know that the above example can be solved directly, but I want to make use of a method (something like Newton's method maybe) for higher dimensional problems, where $x=(x_1,\ldots,x_m)$, and $y=(y_1,\ldots,y_n)$

-
For a numerical answer, I would use Newton's method. (Not that it matters, but many proofs of the IFT and Newton method convergence are very similar.) You need to have a scheme to deal with approaches to points where the Jacobian becomes singular. The IFT gives the derivative of the solution at a solution, so this can be used to compute an initial guess for the perturbed solution. – copper.hat Sep 10 '12 at 17:47

In your example, the IFT guarantees the existence of a $C^1$ function $y$ defined in the neighborhood of $\frac{1}{\sqrt{2}}$ such that $f(x,y(x)) = 0$. Furthermore, $\frac{\partial y(\frac{1}{\sqrt{2}})}{\partial x} = -(\frac{\partial f(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}))}{\partial y})^{-1} \frac{\partial f(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}))}{\partial x} = -1$.

To solve $f(\frac{1}{\sqrt{2}}+0.1, y) = 0$, use Newton's method starting from the initial estimate $y_0 = y(\frac{1}{\sqrt{2}}+0.1) \approx y(\frac{1}{\sqrt{2}})+\frac{\partial y(\frac{1}{\sqrt{2}})}{\partial x} 0.1 = \frac{1}{\sqrt{2}}-0.1$.

This converges quickly (quadratically, of course):

y_1 = 0.590635214224
y_2 = 0.59040553564
y_3 = 0.590405490966
y_4 = 0.590405490966


Here is the Python script that produced the above...

from math import *

# initial values...
x = 1/sqrt(2.0)+0.1
y = 1/sqrt(2.0)-0.1

# update y using Newton's method...
for k in range(1, 5):
y = y - (pow(x, 2.0)+pow(y, 2.0)-1)/(2.0*y)
print "y_%s = %s" % (k, y,)

-
I believe in the 4th line you mean $y_0= \dots = \frac{1}{\sqrt{2}}-0.1$. Is that correct? For the next step do you compute the derivative at $y_0$? But that is not a point on the graph any more, doesn't that make a problem? – Keivan Sep 10 '12 at 22:59
@Keivan: Thanks for catching that! – copper.hat Sep 10 '12 at 23:37
@Keivan: No, you compute the derivative at the current point. – copper.hat Sep 10 '12 at 23:38
Well, that is my problem. I am computing the derivative on some point which is not on the curve any more. Maybe I should go and study the multivariable Newton's method again. I think I'm missing something big here. Thanks for the help. – Keivan Sep 11 '12 at 3:06
@Keivan: If the point is on the curve $f(\frac{1}{\sqrt{2}}+0.1,y) = 0$, then you are finished, so until then, you must be 'off the curve' - the whole point of using Newton's method is to 'get back on the curve'. – copper.hat Sep 11 '12 at 4:58

The IFT tells you that you can solve for $y$ as a function of $x$ in a neighborhood of $a$ in such a way that $y(a)=b$. To approximate $y(a+\delta)$ you can use the Taylor expansion of $y$ around $x=a$. For this you need to know the derivatives of $y$ with respect to $x$ at $x=a$, which can be computed by implicit differentiation.

In your example you want $$y\Bigl(\frac{\sqrt2}{2}+0.1\Bigr)\approx y\Bigl(\frac{\sqrt2}{2}\Bigr)+\frac{dy}{dx}\Bigl(\frac{\sqrt2}{2}\Bigr)\times0.1=\frac{\sqrt2}{2}+\frac{dy}{dx}\Bigl(\frac{\sqrt2}{2}\Bigr)\times0.1$$ To compute the derivative we differentiate the equation $f(x,y)=0$: $$2\,x+2\,y\frac{dy}{dx}=0\implies\frac{dy}{dx}=-\frac{x}{y}\implies\frac{dy}{dx}\Bigl(\frac{\sqrt2}{2}\Bigr)=-1.$$

-
Thanks, but then how do you make the approximation better? By calculating the second, third ... derivatives, or by iteration? If the latter, I'm not sure how to iterate it. – Keivan Sep 10 '12 at 19:54
If you use the Taylor expansion, by computing higher derivatives. Another possibility is to use the Newton-Raphson method. In each iteration you would use implicit differentiation to compute the derivatives. – Julián Aguirre Sep 10 '12 at 20:22
Here is what I don't understand, while using the Newton's method, after the first iteration what shall I use as the initial point in the second iteration? i.stack.imgur.com/6lNHE.jpg – Keivan Sep 10 '12 at 20:48