# How to find the minimum of $f(a_1, b_1, \ldots , a_m, b_m) = \sum_{j=1}^{n} (y_j - \sum_{k=1}^{m} a_kx_j^{b_k})^2$?

$$f(a_1, b_1, \ldots , a_m, b_m) = \sum_{j=1}^{n} (y_j - \sum_{k=1}^m a_kx_j^{b_k})^2$$ $$2m < n$$

$x$ and $y$ are constants, and $a$ and $b$ are variables to find.

I took deviation out of it and somehow solved it and found when it's $0$, but it didn't ended as the minimum. (There was variables that makes the value of $f$ smaller) It's hard for me to work on function I don't know much about.

Note: I am trying to find the variables for the equation which best fits a data, but I can't fit it like $y_1 = a_1x_1^{b_1} + \cdots,\ldots$ because I have more variables than data.

• what is x and y? this is all very unclear. – john Jul 22 '15 at 20:45
• @john They are constants. (Constant series) – LyingOnTheSky Jul 22 '15 at 20:46
• Presumably the bottom-line answer sought here is a function of $(x_j,y_j),\quad j=1,\ldots,n$. If it were just a matter of finding the value of $(a_1,\ldots,a_n)$, then the answer would depend in a linear way on that vector. But finding the exponents $(b_1,\ldots,b_n)$ introduces non-linearity. However, since $m>n$, shouldn't we be able to fit the data points $(x_j,y_j),\quad j=1,\ldots,n$ exactly, making the sum of squares zero? ${}\qquad{}$ – Michael Hardy Jul 22 '15 at 20:54
• @MichaelHardy Fixed the $m > n$, I am in lack of sleep here. You are right, when I provided the right $b$ it could find $a$ as it really is. (With the deviation solution) When I had to provide both, it provided bad variables. (Weirdly when I gave $a$ it found $b$, but sometimes it didn't. Some inconsistently.) – LyingOnTheSky Jul 22 '15 at 21:02

Fitting a model such $$y= \sum_{k=1}^{m} a_k\,x^{b_k}$$ using $n$ data points $(x_i,y_i)$ , $n \gt 2m$, is difficult because the model is nonlinear with respect to its parameters (the $b_k$'s) and nonlinear regression require reasonable starting guesses.

As you noticed, the problem is simple if you assign specific values to the $b_k$'s since the problem becomes a simple multilinear regression.

There is brute force solution which uses this fact. Consider the function $$\Phi(b_1,b_2,\cdots,b_m)=\sum_{i=1}^{n} \Big(y_i - \sum_{k=1}^m a_k\,x_i^{b_k}\Big)^2$$ and generate values of the function on a regular grid; you do not need to compute all points because of the symmetry. Set for example $$b_1=b_0+(i_1-1) \Delta b$$ $$b_2=b_1+i_2 \Delta b$$ $$b_3=b_2+i_3 \Delta b$$ $$b_m=b_{m-1}+i_m \Delta b$$ For sure, you need to select $b_0$ and fix $\Delta b$ in order to limit the $b_k$'s to a maximum value.

For each set of the $b_k$'s, you perform the linear regression and you keep the points which make function $\Phi$ lower. When you finished the generation of the grid, you then know the best approximate values (keep track of the corresponding $a_k$'s) and you are ready for the full nonlinear regression.

It is sure that it could be time consuming if $m$ is large but it works. But if $m \leq 4$ it is quite fast and you can automate the process using Excel.