1
$\begingroup$

t --- 0 1 2 3 4 5 6

F(t) 10 15 23 33 45 58 69

Adjust $F$ by a fnction of the type $$g(t) = \frac{100}{1+\alpha e^{-\beta t}}$$ by the discrete least squares method

I'm studying orhotogonal families of polynomials and projection onto subspaces in the context of least squares method.

I think need to see this problem as a projection onto some subspace and use some inner product but I'm lost.

UPDATE:

Shouldn't the function $g(t)$ be a member of a vector space? I tried applying $\ln$ to see if I'd get something from a vector space but it also won't work

$\endgroup$
3
  • $\begingroup$ You can solve the problem in a simpler manner. $\endgroup$ Apr 7 '19 at 5:01
  • $\begingroup$ @ClaudeLeibovici does it involve least squares method? I need to use it $\endgroup$ Apr 7 '19 at 19:37
  • $\begingroup$ Yes ! The problem can be easily solved using standard least squares methods with anything else. I shall try to make an answer i that spirit. $\endgroup$ Apr 8 '19 at 3:35
4
$\begingroup$

Forgetting (projection/orthogonal families of polynomials), the problem is quite easy to solve using standard nonlinear regression.

As usual, we need good or at least consistent estimates of parameters $(\alpha, \beta)$ and these can be obtained by a linearization of the model. $$g = \frac{100}{1+\alpha e^{-\beta t}} \implies \color{red}{y}=\log \left(\frac{100}{g}-1\right)=\log(\alpha)-\beta\,t=\color{red}{a+b t}$$

Consider the data to be $$\left( \begin{array}{ccc} t & g & y=\log \left(\frac{100}{g}-1\right) \\ 0 & 10 & +2.197225 \\ 1 & 15 & +1.734601 \\ 2 & 23 & +1.208311 \\ 3 & 33 & +0.708185 \\ 4 & 45 & +0.200671 \\ 5 & 58 & -0.322773 \\ 6 & 69 & -0.800119 \end{array} \right)$$ A preliminary linear regression leads to $$\begin{array}{clclclclc} \text{} & \text{Estimate} & \text{Standard Error} & \text{Confidence Interval} \\ a & +2.21599 & 0.01226 & \{+2.18195,+2.25003\} \\ b & -0.50409 & 0.00340 & \{-0.51353,-0.49465\} \\ \end{array}$$ corresponding to $R^2=0.999878$ which is already very good.

This gives as estimates $\alpha=e^a=9.17046$ and $\beta=-b=0.50409$.

Now, we can start the nonlinear regression and obtain $$\begin{array}{clclclclc} \text{} & \text{Estimate} & \text{Standard Error} & \text{Confidence Interval} \\ \alpha & 9.22336 & 0.13438 & \{8.85027,9.59645\} \\ \beta & 0.50576 & 0.00350 & \{0.49603,0.51549\} \\ \end{array}$$ corresponding to $R^2=0.999972$ which is very good. PLease, notice how good are the initial estimates.

Below are reproduced the data as well as the predicted values $$\left( \begin{array}{ccc} t & g & g_{pred} \\ 0 & 10 & 9.782 \\ 1 & 15 & 15.24 \\ 2 & 23 & 22.97 \\ 3 & 33 & 33.08 \\ 4 & 45 & 45.05 \\ 5 & 58 & 57.62 \\ 6 & 69 & 69.27 \end{array} \right)$$

If we had in advance known that the model was good (based on physics for example) and the data in small errors (because of accurate measurements), we could have skipped th first step and used the first and last data points to estimate $(\alpha, \beta)$

$$10=\frac {100}{1+\alpha} \implies \alpha=9$$ $$69=\frac {100}{1+9 e^{-6\beta}}\implies \beta=\frac{1}{6} \log \left(\frac{621}{31}\right)=0.499557$$

$\endgroup$
4
  • $\begingroup$ +1 for the detailed answer and discussion with me. $\endgroup$
    – farruhota
    Apr 8 '19 at 10:54
  • 1
    $\begingroup$ @farruhota. This was my pleasure ! Computing the $SSQ$ from your table gives $0.568$ while, from mine $0.333$. $\endgroup$ Apr 8 '19 at 11:14
  • $\begingroup$ If I didn't round, mine would be $0.356$. Agreed, still more than yours. When you say "Now, we can start the nonlinear regression", are you minimizing $SSQ=\sum_{i=0}^6 \left(g_i-\frac{100}{1+\alpha e^{-\beta t}}\right)^2$ (for example, using $\alpha=\beta=0$ as starting values, the Excel solver gives $\alpha =9.223294081; \beta =0.505758705$)? How do you iterate it from the first step (linear regression)? $\endgroup$
    – farruhota
    Apr 9 '19 at 12:25
  • $\begingroup$ @farruhota. This problem is very simple. In general, use optimization or Newton-Raphson for solving the partial derivatives equal to zero. $\endgroup$ Apr 9 '19 at 17:09
3
$\begingroup$

Make the transformations: $$g(t) = \frac{100}{1+\alpha e^{-\beta t}} \iff \alpha e^{-\beta t}=\frac{100}{g(t)}-1 \iff \underbrace{\ln \left(\frac{100}{g(t)}-1\right)}_{y(x)}=\underbrace{-\beta t}_{ax}+\underbrace{\ln \alpha}_{b}$$ Hence: $$\begin{array}{c|r|r} &x&y(x)&xy&x^2\\ \hline &0&2.20&0.00&0\\ &1&1.73&1.73&1\\ &2&1.21&2.42&4\\ &3&0.71&2.13&9\\ &4&0.20&0.80&16\\ &5&-0.32&-1.60&25\\ &6&-0.80&-4.80&36\\ \hline \text{Total}&21&4.93&0.68&91\\ \end{array}\\ \begin{align}a&=\frac{\sum xy-\frac{\sum x \sum y}{n}}{\sum x^2-\frac{(\sum x)^2}{n}}=\frac{0.68-\frac{21\cdot 4.93}{7}}{91-\frac{21^2}{7}}=-0.5\\ b&=\bar{y}-a\bar{x}=\frac{4.93}{7}-(-0.5)\frac{21}{7}=2.2\\ \ln \alpha&=b=2.2 \Rightarrow \alpha =9.03\\ \beta &=-a=0.5\end{align}$$ So, the final answer: $$g^*(t) = \frac{100}{1+9.03 e^{-0.5t}}\\ \begin{array}{c|c|c} t&g(t)&g^*(t)\\ \hline 0&10&9.97\\ 1&15&15.44\\ 2&23&23.14\\ 3&33&33.17\\ 4&45&45.00\\ 5&58&57.43\\ 6&69&68.99 \end{array}$$

$\endgroup$
8
  • $\begingroup$ You must take care that this is a first step since what is measured is $g$ and not any of its possible transforms. $\endgroup$ Apr 8 '19 at 4:40
  • $\begingroup$ @ClaudeLeibovici, thank you for commenting. Am I not measuring $g$? I transformed and relabeled, which is the linearization. We get the same results except rounding discrepancies. $\endgroup$
    – farruhota
    Apr 8 '19 at 5:11
  • $\begingroup$ This is exactly what I wrote. You transformed $g$ ! Linearization (as we both did) is very good to get estimates of the parameters. Then, you must use $g$ by itself. This case was not bad because of very marginal errors. $\endgroup$ Apr 8 '19 at 5:15
  • $\begingroup$ Yes, $\alpha, \beta$ are estimates of the population parameters calculated from sample data of $7$ observations. Sorry, I’m not getting my mistake if any. $\endgroup$
    – farruhota
    Apr 8 '19 at 5:51
  • $\begingroup$ And by calculating $g^*(t)$ (your $g_{pred}$) it is calculated a point estimate, not interval estimate. $\endgroup$
    – farruhota
    Apr 8 '19 at 5:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.