$A,B$ coefficients for maximal convergence 
In order to approximate the solution to the equation $f(x) = x^2-2$ we use the following iteration:
  $$x_{n+1} = x_n + A\frac{x_n^2 -2}{x^n} + B \frac{x_n^2-2}{x_n^3}$$ 
  Find $A,B$ for maximal rate of convergence.

So first we can easily see that it is a proper function since if we apply $\sqrt 2$ for $x_n$ we indeed get $x_{n+1} = \sqrt 2$.
Now, I've understood that in order to get a maximal rate of convergence one should check when the derivatives are vanished. Is that the case here? What exactly the criteria? 
I'd be glad for guidance.
Thanks!
 A: 
By varying the coefficients $A$ and $B$, we can achieve some mathematical convenience, this is called method of undetermined coefficients.

Let $x_{n}=\sqrt{2}+e_{n}$, then
\begin{align*}
  x_{n+1} &= x_{n}+A\frac{x_{n}^{2}-2}{x_{n}}+B\frac{x_{n}^{2}-2}{x_{n}^{3}} \\
  &= \sqrt{2}+e_{n}+A\frac{(\sqrt{2}+e_{n})^{2}-2}{\sqrt{2}+e_{n}}+
      B\frac{(\sqrt{2}+e_{n})^2-2}{(\sqrt{2}+e_{n})^3} \\
  &= \sqrt{2}+e_{n}+A(\sqrt{2}+e_{n})+\frac{B-2A}{\sqrt{2}+e_{n}}-
     \frac{2B}{(\sqrt{2}+e_{n})^{3}} \\
  &= \sqrt{2}+e_{n}+A(\sqrt{2}+e_{n})+(B-2A)
     \left(
       \frac{1}{\sqrt{2}}-\frac{e_{n}}{2}+
       \frac{e_{n}^{2}}{2\sqrt{2}}-\frac{e_{n}^{3}}{4}+\ldots
     \right) \\
  & \quad  -2B
    \left(
       \frac{1}{2\sqrt{2}}-\frac{3e_{n}}{4}+
       \frac{3e_{n}^{2}}{2\sqrt{2}}-\frac{5e_{n}^{3}}{4}+\ldots
     \right) \\
  &= \sqrt{2}+(1+2A+B)e_{n}+\frac{2A+5B}{2\sqrt{2}}e_{n}^{2}+
     \frac{2A+9B}{4}e_{n}^{3}+\ldots
\end{align*}
To optimize the rate of convergence, we make the linear and quadratic terms in $e_{n}$ vanished:
$$
\left \{
  \begin{align*}
  1+2A+B &= 0 \\
  2A+5B &= 0
\end{align*}
\right.$$
On solving, we have
$$(A,B)=\left( -\frac{5}{8},\frac{1}{4} \right)$$
Substitute back into $x_{n+1}$, we obtain cubic convergence:
$$e_{n+1} \approx \frac{e_{n}^3}{4}$$
