Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given the input-output relation:

$ \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} =p_1 \begin{pmatrix} p_2 & p_3 \\ p_4 & p_4 \end{pmatrix} * \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} + \begin{pmatrix} p_5 \\ p_6 \end{pmatrix} $

with $p_1-p_6$ parameters.I want to minimize the least square error using Gauss-Newton method. Suppose we have 100 measurements. My question is about calculation and size of residual vector.

$ r_i = output - f(input,parameters) \\ \begin{pmatrix} r_1 \\ r_2 \end{pmatrix} = \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} - \Bigg( p_1 \begin{pmatrix} p_2 & p_3 \\ p_4 & p_4 \end{pmatrix} * \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} + \begin{pmatrix} p_5 \\ p_6 \end{pmatrix}\Bigg) $

In order to calculate minimised-error parameters, we have:

$ p_{i+1}=p_i+\Delta \\\Delta=(J_f^TJ^T)^{-1}J_f^Tr_i $

the size of each is as follows:

$ input vector :100*2\\ output vector :100*2\\ r:\quad\quad\quad\quad\quad100*2\\ J_f : \quad\quad\quad\quad100*6\\ p_i:\quad\quad\quad\quad\quad6*1 (six \quad parameters) $

As you can see, the size of $\Delta$ would be 6x2 that seems not consistent with $p_{i+1}$

Now is my residual vector calculation process right? if yes how can I compute parameter's vector? And if not what is the correct answer?

Thank you so much.

share|cite|improve this question

Let me correct your Gauss-Newton step:


In terms of matrix sizes:

$$\left([100\times 6]^{T}[100\times 6]\right)^{-1}[100\times 6]^{T}[100\times 1]=\\ \left([6\times100][100\times 6]\right)^{-1}[6\times 100][100\times 1]=\\ ([6\times6])^{-1}[6\times 1]=\\ [6\times 6][6\times 1]=\\ [6\times 1]$$

Hence the $\Delta$ is a vector with 6 rows which is of the same size as $p_{i}$ and you can compute $p_{i+1}=p_{i}+\Delta$.

You can take a look here for derivation of the Gauss-Newton step.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.