Compute Gradient from Jacobian I have some trouble understanding a formula from a report :
https://www.samba.org/tridge/UAV/madgwick_internal_report.pdf
It is formula (20) (Page 7). Could you tell me where it comes from? 
I can't find anything that resembles in litterature... 
Here is an image of the formula:
http://www.les-mathematiques.net/phorum/addon.php?4,module=embed_images,url=http%3A%2F%2Fs21.postimg.org%2Ft89ej68k7%2Fpourlesmaths.png
The author claims that "Equation (20) computes the gradient of the solution
surface defined by the objective function and its Jacobian"and I don't even understand what he means by gradient since f is a function that goes from R^4 into R^3.
Thanks in advance for your answer
 A: I was stuck on this also, but I think I figured this out and I hope I can
explain clearly without being able to type in math (I don't know how).

In equation 15 he defines what he calls his objective function f, 
which is a 3-dimensional function of 4 variables; i.e. f is the column vector

f = [fx(q1,q2,q3,q4), fy(q1,q2,q3,q4), fz(q1,q2,q3,q4)]'  (' means transpose)

However, I think this is not really the objective function he's using; 
rather he is using the real-valued function

F(q1,q2,q3,q4) = ||f||^2 = fx^2 + fy^2 +fz^2

The gradient of F, grad-F, is 

grad-F = [dF/dq1, dF/dq2, dF/dq3, dF/dq4]'

(the "d's" in the above should be script d's to mean partial derivatives, 
but I don't know how to type those here...).

So

grad-F = [ 2fx dfx/dq1 + 2fy dfy/dq1 + 2fz dfz/dq1
           2fx dfx/dq2 + 2fy dfy/dq2 + 2fz dfz/dq2
           (similar for /dq3)
           (similar for /dq4) ]

This can be written as

grad-F = [ dfx/dq1  dfy/dq1  dfz/dq1   * [ fx
           dfx/dq2  dfy/dq2  dfz/dq2       fy
           dfx/dq3 ...                     fz ]
           ... ]

The left-hand 4x3 matrix in the above is just the transpose of the 
3x4 Jacobian, so can write this as:

grad-F = J' * f

Compare with Madgwick's eq. 20. I think you'll agree this is what
he was getting at.

A: yeah, got the same question :), this is what i think:
we use gradient descent algorithm.
 A) If your target function is a single function (whatever variables
    number is) it is simple case described in all literature - you compute
    gradient without this multiplication from equation (20).
B) If your target function is composed of more functions - it is a vector of function - like in this algorithm by Madgwick :
d - s = [dx dy dx] - [sx sy sz]  -> [dx-sx   dy-sy   dz-yz]  [f1  f2  f3]
In this case you also compute gradient for each function, wchich forms a vector, which is a Jacobian matrix.
but how you want to optimize all functions at once???  when you optimzie your step for  (dx-sx), you could ruin value of other function e.g. (dz-sz) - (not sure if this is true :)
How can you solve this?   you need to multiply each target function component by its value, so the step for those which are already optimized (d-s ~= 0)(x,y or z) would have weight around 0.
Thats mean this multiplication is necessary for this algorithm to produce some solution, but it is actually synthetic, artifical. I feel like this is synthetic.
I didnt found any serious literature that states that ,,if you use vector target function you need to multiplu the gradient vector (jacobian matrix) by target function values vector'', so i cant provide proof for my elaborate but this is how i understand this. I suppose it is preety much fair.  
Maybe this is so obvious nobody mention this. It is popular approach.
regards :)
