Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have the following function:

$$M = a(\log_{10}W-2.5)+b$$

I also have a set of data with actual measured values of $W$ and $M$ (each have individual $\pm$ errors). Here's a small sampling of the data:

W(x)    M(y)        M_model
245     -19.59      -19.05  
155     -16.64      -17.76
314     -20.26      -19.75
351     -20.78      -20.07
192     -17.96      -18.37
...     ...         ...

Using $W$ and $M$ values from the dataset, I did a non-linear least squares fit (using Matlab's optimtool)and a $y = ax + b$ form where $x = \log_{10}(W)-2.5$. Found values of $a$ and $b$, and then calculated $M_{model}$ in the table above.

$$a = -6.359$$ $$b = -19.83$$

How do I find what uncertainty there might be in $a$ (slope) and $b$ (intercept)? I need the function in the format:

$$M = (-6.359\pm c)(\log_{10}W-2.5)+(-19.83\pm d)$$

Update I have tried using LINEST() in Excel which provides me with the errors, but it also provides me with slightly different $a$ and $b$ values too. Is there a more robust way of calculating these uncertainties?

Update I have tried Michael's solution below, and that gives me one figure, substituting his $x$ for my $M$. I think that gives me the $\pm slope$, but I am not sure if that is correct. Also not sure how to get $\pm intercept$ as I only have one set of residuals (for $M$).

share|cite|improve this question
up vote 1 down vote accepted

Supposing (unobservable) errors (as opposed to observable residuals) are independent and identitcally distributed, and normally distributed with expectation $0$ and variance $\sigma^2$, then the least-squares estimate $\hat a$ of the slope $a$ satisfies $$ \hat a \sim N\left(a, \frac{\sigma^2}{\sum_i (x_i-\bar x)^2} \right) $$ where $\bar x=(x_1+\cdots+x_n)/n$.

Now let $\hat\sigma^2$ be the estimate of $\sigma^2$ that comes from dividing the sum of squares of observed residuals by the degrees of freedom, in this case $n-2$. Then $$ (n-2) \frac{\hat\sigma^2}{\sigma^2}\sim \chi^2_{n-2}, $$ and (believe it or not!) this random variable is independent of $\hat a$.

Therefore $$ t = \frac{\hat a - a}{\hat\sigma/\sqrt{\sum_i(x_i-\bar x)^2}} $$ has a $t$-distribution with $n-2$ degrees of freedom.

So $$ \hat a \pm A \frac{\hat\sigma}{\sqrt{\sum_i(x_i-\bar x)^2}} $$ are the endpoints of a confidence interval for $a$. The number $A$ you get from one of the standard tables for the $t$ distribution, and depends on what confidence level you want.

Two thoughts on the updated version of the question:

  • The estimates of slope and intercept are correlated with each other (except when the mean of the $x$-values is $0$. If the line is $y=ax+b$, then we'd have a confidence region for the pair $(a,b)$ that would be an ellipse whose axes are not parallel to the $x$- and $y$-axes. So it would be somewhat misleading to express them separately as $\hat a\pm\text{something}$ and $\hat b\pm\text{something}$. However, if the line is expressed point-slope form, as $y-\bar y=a(x-\bar x)$, where $\bar y$ and $bar x$ are the mean values, then $\bar y$ would be uncorrelated with $\hat a$.
  • All this assumes the $y$s are random variables and the $x$s are fixed. That can make some sense if all results are regarded as conditional expected values, given the $x$ values. It's pretty conventional to do things that way, but possibly in some situations it's not appropriate.
share|cite|improve this answer
Thanks for that. I'm not familiar with the notation, but it's making some sense. Wondering though, what is $\hat\sigma$ – Carl Apr 18 '13 at 16:45
$\hat\sigma$ is an estimate of $\sigma$ based on the observed data. As I wrote: "Now let $\hat\sigma^2$ be the estimate of σ2 that comes from dividing the sum of squares of observed residuals by the degrees of freedom, in this case $n-2$." If the least-squares line is $y=\hat ax+\hat b$, then the $i$th residual is $y_i-\hat y_i$ $=y_i - (\hat ax_i+\hat b)$. See Errors and residuals in statistics. ${}\qquad{}$ – Michael Hardy Apr 18 '13 at 19:52
Thanks for that. Just wanted to check that in your last equation you intended $\hat\sigma$ and not $\hat\sigma^2$. So $\hat\sigma$ is $\sqrt{\hat\sigma^2}$ – Carl Apr 19 '13 at 10:13
I'll take a look at that..... – Michael Hardy Apr 24 '13 at 14:45
I've added some comments to my answer. – Michael Hardy Apr 24 '13 at 14:59

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.