General nonlinear least squares?

Question

I'm looking to solve some kind of generalized nonlinear least squares problem, I think.

So for some background, lets say I have an ordinary nonlinear least squares problem. That is, a set of data points ${(x_i,y_i)}$, an objective function $f(x,\beta)$ where $\beta$ is a vector of parameters that I wish to find out in order to minimize $\sum [y_i - f(x_i,\beta)]^2$. Now I can use some appropriate algorithm, e.g. Levenberg–Marquardt and I'm done.

However, my problem now is that I have several sets of data points,say $(x_{ij},y_{ij})$ (my objective function is the same for all sets, tough) and I want to find a single vector $\beta$ that minimizes (in a least-squares sense, say) the residual over all these data sets. So how do I go about solving this? Is the best way just to combine all my sets into one and then do a normal nonlinear least squares fit?

EDIT: More info about the problem

I have a bunch of "components", lets call them A, B, and C. I also have the functional form $f(x,\beta)$ that each component should follow, but I need to figure out the vectors $\beta_A,\beta_B,\beta_C$. Now each set of data points does not contain a component in isolation, but a combination of them, e.g. one data set for A+B, another set has A+B+C, a third B+B+C, and so on. Note that the "+" in the previous sentence is not exact; I do know that the contributions are approximately additive. So from these data sets I want to figure out an approximation of the contribution of each component.

EDIT #2: "Solution"

In the end I just did the simple approach and combined all my sets into one. My Jacobian ended up looking a bit funny (sort-of blocked) but it all worked out decently in the end. Thanks for all the suggestions.

Answer 1

What distinguishes one data set from another? If you want the same $\beta $ to fit all the data, yes you can just combine all the data sets. The sum of the squared errors is just a sum over points and whether you do one sum over all the points or several sums over subsets and add them the result is the same.

Sometimes in these cases there are some parameters in $\beta $ that should be the same for all data sets and some that are allowed to change from one set to another. Then you can view your data points as $(x_i,y_i,j)$ where $j$ is the dataset and make copies of the parameters in $\beta $ that are allowed to vary between the sets, then do one global fit.

Answer 2

If you have access to IEEE maybe this can help you a little or provide fresh ideas. If you don't let me know and I can send it to you. It is kind of new and I attended to the speakers talk. He had a complicated style but the gist is visible. The word you are looking for is robustness by the way, :).