How can I fit a set of data points to a hyperbola, a square root function or a logarithmic function? Hello StackExchange community!
I am currently a 12th grader who has to research statistics on a post-high school level.
The topic that I was assigned is least-squares regression. So far I have read about and understood the concept behind linear least-squares regression. I then needed to gather data for regression analysis and now that I have gathered sufficient data, I know that the data does not follow a linear model. I, however, am really struggling to understand any other forms of curve fitting apart from data that follows a linear relationship, so here is the situation:
I have a set of points in the form of (x,y) that l can either fit by a hyperbola of the form $y=k/x$, a logarithmic function of the form $y=a*ln(cx+d)+g,$ or a square root function of the form $y=a(\sqrt {bx+c})+d$. All that I have to do using the least-squares method.
Could someone please explain to me (step-by-step and in easy terms) how I would do that? An example would be greatly appreciated! I already read something about linearization and then using linear regression, but frankly speaking I don't quite understand that either, so it'd be great if you could explain that too. You would help out a high-schooler in great despair haha.
Thank you very much in advance!
(P.S I apologize for any semantic or grammatic mistakes. English is not my native language.)
 A: Linearization is a blanket term which refers to modifying the independent variable so it relates linearly to the dependent variable. The idea is to linearize the data via the three potential models, and then try least squared regression on all of them.
For example, to linearize the hyperbola, introduce a new independent variable $z=\frac1x$. Why is this helpful? Well, note that in the hyperbola model, $y=kz$. So, you can do LS regression on $y$ and $z$ to fit the hyperbolic model.
Likewise, we can linearize $x$ into $z$ to fit the other models. 
To linearize the logarithm, let $z=\ln(cx)$, which makes the logarithmic model $y=a\ln(d)\cdot z+g$.
To linearize the square root, let $z=\sqrt{bx+c}$, which makes the square root model $y=az+d$.
A: Being very trivial, I shall say that, if the model is nonlinear with respect to even a single parameter, you will need nonlinear regression or optimization. The problem is that, most of the time, this requires good or, at least, consistent, estimates of the parameters.
Linearization helps in some case. For example, $y=a e^{bx}$ is easy to linearize but $y=a e^{bx}+c$ is not.
In any manner, you need by the end to use nonlinear regression or optimization since what is measured is $y$ and not any of its possible transforms.
If you have one case, let us try together for a small number of data points you could add to your post (even synthetic data with noise).
Edit
After comments, let us consider the following data set
$$\left(
\begin{array}{cc}
 x & y \\
 1.40 & 0.33 \\
 2.18 & 2.33 \\
 1.74 & 1.27 \\
 3.64 & 2.43 \\
 3.00 & 3.00 \\
 2.66 & 1.65 \\
 5.34 & 3.65 \\
 6.14 & 3.13 \\
 7.00 & 4.00 \\
 7.80 & 3.99 \\
 8.62 & 3.37 \\
 9.16 & 3.91 \\
 0.90 & 3.40
\end{array}
\right)$$ to be fitted according to the model
$$y=\frac{k}{b+x}+c$$ In a prelimnary step, I should fix $b$ at an arbitrary value and define $t_i=\frac{1}{b+x_i}$. This makes the model to be $y=b t+b$ which is a linear regression. For this value of $b$, compute the sum of squares of the residuals $(SSQ)$.
Running for different values of $b$, you should notice that it is a very flat function of  and this is not good sign (already revealed by a scatter plot of the data). Anyway, the minimum value appears around $b=28$ and , at this point, we have $k= -305.56$ and $c=12.25$.
Now, we have all the elements to run the nonlinear regression which will give $b=28.09$, $k= -307.28$ and $c=12.28$ corresponding to $R^2= 0.936$ which is not very good.
A: Could someone explain to you (step-by-step and in easy terms)  how do that?
Whithout wanting to distress you, I would say that is not so simple to be able to explain this wide subject "step-by-step and in easy terms". One have to make effort to learn the basics. 
http://mathworld.wolfram.com/NonlinearLeastSquaresFitting.html
https://en.wikipedia.org/wiki/Non-linear_least_squares
In practice it is important to understand that the fitting cannot be good if the shape of the function to fit is far from the shape of the data. It is of interest to first observe a drawing of the points. For example with your data :

We observe that the shape is not far to linear. So the linear regression is the first to try : $y(x)\simeq ax+b$

Trying to fit the function $y(x)\simeq \frac{k}{x}$ is wasting time because obviously the function is not convenient ($k>0$ for positive $y$ thus decreasing function while increasing is required). 
Instead of $y(x)\simeq \frac{k}{x}$ one can try a more general form of hyperbolic equation :
$$y(x)\simeq\frac{ax+b}{cx+d}\quad\text{with}\quad d=1\quad\text{for non-redondency.}$$
This example allows to show a possible linearisation (among others):
$(cx+1)y\simeq ax+b$
$$a\frac{x}{y}+b\frac{1}{y}+c(-x)\simeq 1$$
A linear regression can be carried out for the coefficients $a,b,c$.
"Linear" means linear with respect of $a,b,c$, of course not wrt the variable terms $\frac{x}{y}\,,\,\frac{1}{y}\,,\,(-x)$.

Note : This is not best fit of $y(x)=\frac{ax+b}{cx+1}$ in the sens of least mean square because it is the best fit for $a\frac{x}{y}+b\frac{1}{y}+c(-x)= 1$ which is not exactly the same criteria of fitting. If we want the least mean squate in strict sens one have to proceed with non-linear regression. This is an iterative process requiring a good initial guess of the parameters. The above values of $a,b,c$ can be taken as very good initial values.
It should be much too long to go into the non-linear regression which suppose to built an algorithm or to use a commercial software.
Case of the function $y(x)=a\sqrt{bx+c}+d$ . It can be lineralized :
$(y-d)^2=a^2bx+a^2c$
$y^2-2d\,y-a^2bx=a^2c-d^2$
$$Ay^2+By+Cx=1$$
$A=\frac{1}{a^2c-d^2}\quad;\quad B=\frac{-2d}{a^2c-d^2}\quad;\quad C=\frac{-a^2b}{a^2c-d^2}$
You can do a linear regression for $A,B,C$. This gives a first result for $a,b,c,d$ in fixing one of them since there is a redondency. If necessary one can improve thanks to a non-linear regression starting from the first values obtained.
I don't continue on this case because I suspect that the function $y(x)=a\sqrt{bx+c}+d$ is not convenient for a good fitting to the given data.
Case of the function $y(x)=a\ln(cx+d)+g$.
Linearisation seems possible but would involve the transformation into an integral equation. Some examples are given in https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales . This is a non-conventional method. More likely go directly to non-linear regression. 
A: I post a second answer because the data provided by the OP changed meamwhile.
The scatter is much larger than before which changes a lot the scope.
For the reccord the linear regression leads to :

HYPERBOLIC REGRESSION :
Claude Leibovici already gave a very good answer in the case of an hyperbolic regression. The next graph is drawn from his results :

The scatter is so large that the non-linear regression doesnt improve the fitting. The Root Mean Square Errors are very close to be the same. The graphs (blue curves) are undistinguishable. So, in this case there is no need for a final non-linear regression. The preliminary approximate as Claude Leibovici did is sufficient.
Moreover by comparing to the above linear regression, one see that the RMSE is almost not improved when going from linear regression to hyperbolic regression. So, in this case of large scatter it is of no interest to fit an hyperbolic function instead of a simple linear function.
NOTE : From inspection, one point: $(0.9,3.4)$ appears as an outlier. Eliminating it changes the conclusion.

An even simpler hyperbolic function could be used with almost the same fitting.
$$y(x)=\frac{a}{x}+c$$

