Constrained parameters in non linear curve fitting I have to fit some data points to the non linear curve defined by:
$$y(x) = a + \frac{b}{(x - c)}$$
with constraints $a,b,c \geq0$
So far I've tried gradient descent with penalty methods but I'm not getting correct results.
At any point $(x_0,y_0)$, my loss function looks like:
$$(y_0 - y(x_0))^2 -r (\log(a) + \log(b) + \log(c))$$
What's the best way to model this?
I was going through Constrained parameters in least square curve fitting, which is similar to my problem. How can I use this approach with the curve I defined?
 A: What I'm going to propose is definitely not common but it will definitely help with data that is not very close to the singularity $c$, since it penalizes them more than the conventional approach to the loss function and therefore makes them more visible to the minimization procedure. As an example, if you want to fit a number of points on the tail of the curve, then finding the value $c$ is certainly unstable, since all the data points you have have a similar loss function and thus the parameters a,b,c will have similar loss functions for a wide range of values, making it hard to distinguish what the right choice is, especially if the problem is non-linear and has several minima.
Consider the loss function
$$L=\sum_i(y_i(x_i-c)-a-b(x_i-c))^2$$
If it is zero, then we recover that the (x,y)'s are on the same curve you wish to use. It is a fortunate coincidence that the minimization problem of this loss function has a unique solution!
$$\frac{\partial L}{\partial a}=\sum_i (y_i(x_i-c)-a-b(x_i-c))=0\\
\frac{\partial L}{\partial b}=\sum_i (y_i(x_i-c)-a-b(x_i-c))(-x_i+c)=0\\
\frac{\partial L}{\partial c}=\sum_i (y_i(x_i-c)-a-b(x_i-c))(-y_i+b)=0$$
We can rewrite these equations in the form (note that $\overline{WXY}=\frac{\sum_i W_iX_iY_i}{N}$):
$$a+b\bar{x}+c\bar{y}-bc-\overline{xy}=0\\
a\bar{x}+b\overline{x^2}+c\overline{xy}-bc\bar{x}-\overline{yx^2}=0\\
a\bar{y}+b\overline{xy}+c\bar{y^2}-bc\bar{y}-\overline{y^2x}=0$$
After a miraculous cancellation of the non-linear term $bc$ we obtain the unique solution
$$b=\frac{AZ-BY}{XZ-Y^2}\\ 
c=\frac{BX-AY}{XZ-Y^2}\\
 a=-b\bar{x}-c\bar{y}+bc+\overline{xy}$$
where I defined:
$$A=\overline{x^2y}-\overline{xy}~\overline{x}~~,~~B=\overline{y^2x}-\overline{xy}~\overline{y} \\
X=\overline{x^2}-\bar{x}^2~~, ~~Z=\overline{y^2}-\bar{y}^2~~, ~~ Y=\overline{xy}-\bar{x}\bar{y}$$
With this now you have a tool to minimize with analytical control. Hope it works!
