Least squares method of a nonlinear function Let $$y(x)=a\cdot\frac{x^c}{b^c}$$
Assume that we have a number of data samples $(x_i,y_i)$.
The objective is to find $a, b,$ and $c$ that minimizes the square error function, i.e.,
$$J(a,b,c) = \sum_{\forall i} (y(x_i)-y_i)^2$$
As always, to minimize, we set the derivatives to 0, i.e.,
$$\frac{\partial J}{\partial a}=0 \iff  2\sum_{\forall i} (a\cdot\frac{x_i^c}{b^c}-y_i)\cdot \frac{x_i^c}{b^c}=0 \iff \sum_{\forall i} (a\cdot\frac{x_i^c}{b^c}-y_i)\cdot x_i^c=0$$
$$\frac{\partial J}{\partial b}=0 \iff  2\sum_{\forall i} (a\cdot\frac{x_i^c}{b^c}-y_i)\cdot c \cdot a \cdot x_i^c \cdot b^{-c-1}=0 \iff \sum_{\forall i} (a\cdot\frac{x_i^c}{b^c}-y_i)\cdot x_i^c=0$$
$$\frac{\partial J}{\partial c}=0 \iff  2\sum_{\forall i} (a\cdot\frac{x_i^c}{b^c}-y_i)\cdot a\cdot\frac{x_i^c}{b^c} \cdot \log\frac{x_i}{b}=0 \iff \sum_{\forall i} (a\cdot\frac{x_i^c}{b^c}-y_i)\cdot x_i^c \cdot \log\frac{x_i}{b}=0$$
As you may have noticed, I'm supposed to get a 3-by-3 system of equations, but due to cancellation, I got 2 equations and 3 unknowns (the 1st and 2nd equations are identical after simplification). 
Does this mean the existence of many solutions? Or did I do some mistake in deriving the equations? Please note that all the parameters are assumed to be positive reals and greater than zero.
Please help. 
 A: The problem is that there are infinitely many least-square solutions. This can for example be seen by noticing that $J(k^ca,kb,c) = J(a,b,c)$ for all $k\not= 0$. We can only constraint the parameter combinations $(a/b^c,c)$ so your fitting-function has one parameter that cannot be constrained. You can get around this problem by instead trying to fit to the functional form $f(x) = a x^c$ instead. With this form you will get a well defined equation-system.
You can also, if wanted, reduce the problem to a standard linear least-square problem by taking $\log$'s which allows you to use the standard well-known formulas for this case directly. If we define $\tilde{y}_i = \log(y_i)$ and $\tilde{x}_i = \log(x_i)$ (assuming for simplicity that $y_i > 0$; otherwise we need to work with the absolute value) and try to fit these points to $y(\tilde{x}) = \alpha \tilde{x}+\beta$ then the least-square solution for $(\alpha,\beta)$ are related to the least-square solution for $(a,c)$ via $c = \alpha$ and $a = e^{\beta}$.
A: *

*The problem statement has redundant parameters; eliminate one parameter. Consider the form
$$
y(x) = \alpha x^{c}
$$
which has two fit parameters, $\alpha$ and $c$. The constant term is
$$
 \alpha = a b^{-c}.
$$

*Use least squares to constrain the constant parameter which appears in a linear form. Using the new trial function from part one, and $m$ measurements $\left\{ x_{k}, y_{k} \right\}_{k=1}^{m}$, the least squares solution is defined as
$$
  \left[
    \begin{array}{c}
       \alpha \\
       c
    \end{array}
  \right]_{LS} 
=
  \left\{ 
  \left[
    \begin{array}{c}
       \alpha \\
       c
    \end{array}
  \right]
\in
  \mathbb{R}^{2}
  \colon
   r^{2} (\alpha,c) = 
   \sum_{k=1}^{m}
   \left(
      y_{k} - \alpha x_{k}^{c}
   \right)^{2}
   \text{ is minimized}
  \right\} 
$$
The minimization criterion
$$
  \frac{\partial}{\partial \alpha} r^{2} = -2 
   \sum_{k=1}^{m}
   \left(
      y_{k} - a x_{k}^{c}
   \right)
x_{k}^{c} 
= 0
$$
provides the constraint
$$
  \hat{\alpha}(c) = \frac{\sum_{k=1}^{m} y_{k} x_{k}^{c}} {\sum_{k=1}^{m} x_{k}^{2c}}.
$$
The two parameter nonlinear least squares problem is now reduced to a one dimensional minimization problem
$$
  r^{2}(\hat{\alpha}(c),c) = r^{2}(c) =    
   \sum_{k=1}^{m}
   \left(
      y_{k} - 
\frac{\sum_{k=1}^{m} y_{k} x_{k}^{c}} {\sum_{k=1}^{m} x_{k}^{2c}}
 x_{k}^{c}
   \right)^{2}.
$$
This solution will be unique.
