I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another.
To explain, given the standard linear least squares problem: $$ \textbf{Y} = \textbf{X} \mathbf{\beta} + \mathbf{\epsilon} $$

where $\textbf{Y}$ is the response vector, $\textbf{X}$ is the design matrix, $\beta$ is the solution vector, and $\epsilon$ is the residual vector - Is there a linear way to implement a 'weight' on a certain function (column) within the design matrix, such that this function (or set of functions) have a higher propensity to fit closer to the data than the other functions?

I have stumbled across weighted linear least squares, but that appears to assign higher propensity to fit certain sections of the response more accurately than the rest of the response via a diagonalized weight matrix (if I am looking at it right). I believe my problem is different.

For instance, given a $\textbf{Y}$ of experimental data such as this:

experimental data

with one function (column) of the design matrix such as this in red:


and other functions which make up the design matrix with much larger derivatives (incommensurate with that of the spectral response of $\textbf{Y}$):


We may end up getting a better linear fit by choosing non zero values for the several small spectral responses over the one larger one, which may make more physical sense (although there has been some error in calculating where the mean is). So we end up with a choice of the red or orange lines given here:

soln choice

Is there a way of making sure that the spectral responses ahve similar underlying profiles, or that small derivative functions prevail in fitting where the experimental data has a small derivative itself?


1 Answer 1


It looks like your first fit is trying to fit all the data to a single spectral line. The problem is the bumps in your data around $6.5$ and $15$. Depending on how you assess the probable error on each point, those bumps are going to pull your fit very hard. This is especially true if you take the error to be something like the square root of the expected number of counts in the bin. I am surprised to see the line shifted that far to the right. I would have expected it to be better centered, but broader than it should be. You might have an error that adds $1$ to the $x$ value in your fitted equation. I would suggest you try fitting some cleaner data and see if it fits properly.

On siemple fix would be to only fit the data from $7$ to $14$. The outside bumps might be noise. Another fix would be to fit your data to the sum of three spectral lines, letting the two new ones take care of the bumps. There may not be enough in the small bumps for this to work-you might get

  • $\begingroup$ I'm sorry, I probably should have been a little more clear about that data. It's more of an example to show an idea. The data shown isn't even fit completely with linear least squares, I just threw it into excel to get the point across. The extra bumps are just random error I added to make it look more like real data. My point is more to find an algorithm that would try to fit each function or column in the design matrix to the data but put more weight on a function that has a derivative closer to that of the data. $\endgroup$
    – chase
    Jun 12, 2015 at 4:57
  • 1
    $\begingroup$ You need to state your question clearly. According to my favorite text, [Numerical Recipes](nr.com), fitting is a creative (idiosyncratic) process. If you want the derivatives to match, you can add variables that are the derivatives and the numeric values taken from your data. The problem is that derivatives magnify the noise, so taking one numerically in a way that means something is difficult. $\endgroup$ Jun 12, 2015 at 5:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .