Least squares for rational function Simple question: If I want to fit data points to a function of the form $y=a+\frac{b}{x}$, is there any reason why I can't use a least-squares approach?
I want to minimize $$E=\sum_{i=1}^{n}(y-a-\frac{b}{x})^2$$
So just set $\frac{\partial E}{\partial a}=\frac{\partial E}{\partial b}=0$ and solve the linear system?
 A: Problem statement
Start with a set of $m$ measurements $\left\{ x_{k}, y_{k} \right\}$ and the model function
$$
 y(x) = a + \frac{b}{x}.
$$
Assume the data are distinct to the point where the system matrix will have full column rank. Assume none of the $x$ values are $0$.
Least squares definition
The least squares solution is defined as 
$$
 (a,b)_{LS} = \left\{
(a,b)\in \mathbb{R}^{2} \colon
\sum_{k=1}^{m} \left( y_{k} - a - \frac{b}{x_{k}} \right)^{2}
\text{ is minimized}
\right\}
$$
Linear system
$$
\begin{align}
  \mathbf{A} a &= y \\
%
\left[
\begin{array}{cc}
 1 & \frac{1}{x_{1}} \\
 \vdots & \vdots \\
 1 & \frac{1}{x_{m}}  \\
\end{array}
\right]
%
\left[
\begin{array}{c}
 a \\
 b  \\
\end{array}
\right] &=
%
\left[
\begin{array}{cc}
 y_{1} \\
 \vdots \\
 y_{m} \\
\end{array}
\right]
%
\end{align}
$$
Linear system solution
The least squares solution for the full column rank problem is
$$
 %
\left[
\begin{array}{c}
 a \\
 b  \\
\end{array}
\right]_{LS}
=
\mathbf{A}^{+} y
$$
One path to this solution is to use the normal equations:
$$
\begin{align}
  \mathbf{A}^{*} \mathbf{A} \, a &= \mathbf{A}^{*}y \\
%
\left[
\begin{array}{cc}
 \mathbf{1} \cdot \mathbf{1} & \mathbf{1}  \cdot \frac{1}{x} \\
 \frac{1}{x} \cdot\mathbf{1} & \frac{1}{x} \cdot \frac{1}{x}  \\
\end{array}
\right]
%
\left[
\begin{array}{c}
 a \\
 b  \\
\end{array}
\right] &=
%
\left[
\begin{array}{cc}
 \mathbf{1} \cdot y \\
 \frac{1}{x} \cdot y
\end{array}
\right]
%
\end{align}
$$
These are exactly the same equations you would get by the minimization process you outlined:
$$
\begin{align}
%
& \frac{\partial}{\partial a} \sum_{k=1}^{m} \left( y_{k} - a - \frac{b}{x_{k}} \right)^{2} = 0, \\
%
& \frac{\partial}{\partial b} \sum_{k=1}^{m} \left( y_{k} - a - \frac{b}{x_{k}} \right)^{2} = 0. \\
%
%
%
\end{align}
$$
The solution is then
$$
\left[
\begin{array}{c}
 a \\
 b  \\
\end{array}
\right]_{LS}
=
\left( \mathbf{A}^{*} \mathbf{A} \right)^{-1} \mathbf{A}^{*} y
$$
(These remarks are an elaboration of the points made by @David.)
