Jacobian of $A^{-1}b$ I need to calculate the Jacobian $\frac{df}{dx}$ of $f=A^{-1}b$ where $A$ and $b$ are a function of $x$, the variable towards to differentiate.
I thought 
$$\frac{df}{dx} = \frac{dA^{-1}}{dx} b  + A^{-1}\frac{db}{dx}$$
by the product rule, 
and since $A^{-1}A=I$,
$$\frac{dA^{-1}}{dx} = A^{-1} \frac{dA}{dx}  A^{-1}.$$
Now the last thing i thought is 
$$\frac{dA}{dx} = \frac{dA}{dx_1} + \frac{dA}{dx_2} + \frac{dA}{dx_3} + \cdots $$
The last step is to calculate the Jacobian of a matrix. However, if I try this for a simple example, I get a wrong answer. Can anyone see where I make the mistake? How can I calculate the Jacobian of $f=A^{-1}b$ correct if I cannot analytically invert $A$ (I can only do that numerically)?
 A: To elaborate on my comment, the problem is that if you want to stay in the usual linear algebra notation, it's not really clear what kind of object the derivative of a matrix $A$ with respect to a vector $x$ really is, or how to multiply it with another vector $b$. You can either redo everything using index notation, or give up a little bit of abstraction and work componentwise, as follows.
Recall that the Jacobian of $f(x)$ is a matrix whose $j$th column is $\partial f/\partial x_j$. Expanding this out, we have
$$\frac{\partial f}{\partial x_j} = \frac\partial{\partial x_j}(A^{-1}b) = A^{-1}\frac{\partial b}{\partial x_j} + \frac{\partial A^{-1}}{\partial x_j}b.$$
Here $\partial A^{-1}/\partial x_j$ is the derivative of a matrix with respect to a scalar, so it's still a matrix, and we know how to work with those. Using the fact that
$$\frac{\partial A^{-1}}{\partial x_j} = -A^{-1}\frac{\partial A}{\partial x_j}A^{-1}$$
(you have the wrong sign in the equation in your question), we get
$$\frac{\partial f}{\partial x_j} = A^{-1}\frac{\partial b}{\partial x_j} - A^{-1}\frac{\partial A}{\partial x_j}A^{-1}b.$$
We can pull this partially back into matrix form, because $\partial b/\partial x_j$ are just the columns of the Jacobian of $b(x)$. However, there's not much we can do about the second term without bringing tensors into the picture. So what we get is
$$J_f = A^{-1}J_b - \begin{bmatrix}A^{-1}\frac{\partial A}{\partial x_1}A^{-1}b & A^{-1}\frac{\partial A}{\partial x_2}A^{-1}b & \cdots & A^{-1}\frac{\partial A}{\partial x_n}A^{-1}b\end{bmatrix}.$$
