# Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$y = f(x_1, x_2, x_3,\ldots, a_1, a_2, a_3,\ldots)$$ Where the $a_1, a_2,\ldots$ are fitting parameters.

I can use least squares to find the least-squares best fitting parameters for each individual set of observations, but as it happens, this isn't what I really need. Instead it is thought that the physical meaning behind the fitting parameters is such that for all five sets of observations some of the fitting parameters, say $a_3$ & $a_4$, should be the same, but the $a_1$ and $a_2$ should be different for each set.

So I need to find a "globally" "optimal" (in some sense) $a_3$ and $a_4$, and then the best fit $a_1$ & $a_2$ for each observation set.

Can somebody suggest a way to approach this, and if there are standard techniques for it?

I'm aware that people in some fields approach this by iteratively trying to find the best fit $a_3$, $a_4$ by treating the five sets of observations as one big set, and then they fix those parameters and then split into five sets again and then solve for the $a_1$ & $a_2$. And more elaborate iterative approaches are used, but I've never seen this described in a text on numerical methods, and the method is used by metallurgists and other professionals who may not be appreciating how meaningful, or not, their algorithm to solving the problem. So I have doubts about using that approach (are they just getting the numbers they want to see, rather than physically meaningful numbers?)- unless there is good reason to believe that it is effective?

Thanks, Neil

Here's what I would do. (I am assuming for the moment your $f$ is linear, so we can use OLS. If that's not the case, you have to use nonlinear LS or something appropriate.)
I would throw your 5 sets of data together. Now if you estimate your $a$ coefficients on that big data set, your $a_3,a_4$ will be "global" in the sense you want. But so will be $a_1$ and $a_2$, which you don't want.
What we instead want is that these two are optimal for each subset, so we need to estimate those two for each subset, $10$ coefficients overall (call them $a_{j1}$ and $a_{j2}$, for $j=1,...,5$). We can do this with interactions. Define dummies $d_{ji}$, where $d_{ji}=1$ if and only if observation $i$ is from set of observation $j$, and otherwise zero. Hence, you estimate with OLS $$y_i=\sum_{j=1}^5 (a_{j1} d_{ji} x_{i1}+a_{j2} d_{ji} x_{i2})+a_3 x_{i3}+a_4 x_{i4}+\varepsilon_i.$$ Suppose observation $i$ is from set $3$, then $d_{1i}=d_{2i}=d_{4i}=d_{5i}=0$, and $d_{3i}=1$. Hence, only coefficients $a_{31}$ and $a_{32}$ are influenced by this observation, not the coefficients that are supposed to capture the other sets of observation.
How to estimate this exactly? You will have to prepare the data. Just use your two variables $x_{1}$ and $x_{2}$ (the ones for which you want a different coefficient depending on the set), and create 5 new variables from $x_1$ such that the first is $d_{1i}*x_{i1}$, the second $d_{2i}*x_{i1}$ and so on. Do the same for the second variable and then just estimate the $2*5+2$ coefficients with OLS. Done.