Derivative of Inverse of sum of matrices Given is the function $f : \mathbb{R}^p \to \mathbb{R}$ with
$$
f(x) = q(x)^{\top} G^{-1} q(x)
$$
where $G = A + x_1 B_1 + \ldots + x_p B_p$.

*

*The matrices $A, B_1, \ldots, B_p \in \mathbb{R}^{n \times n}$ are all symmetric positive definit.


*$q: \mathbb{R}^n \to \mathbb{R}^n$ and the Jacobian $\nabla q$ is known.
Is it possible to derive a closed form for $\nabla f$? For me, the hard part is
$G^{-1}$. Any hints or suggestions are really appreciated!
 A: $$\frac1{\Delta x}\big[(A+(x+\Delta x)B)^{-1}-(A+xB)^{-1}\big]=$$
$$\frac1{\Delta x}(A+xB)^{-1}(A+xB)\big[(A+(x+\Delta x)B)^{-1}-(A+xB)^{-1}\big]
(A+(x+\Delta x)B)
(A+(x+\Delta x)B)^{-1}$$
$$=-(A+xB)^{-1}B
(A+(x+\Delta x)B)^{-1}\rightarrow-(A+xB)^{-1}B(A+xB)^{-1}$$ as $\Delta x\rightarrow0$.
So that $\partial f/\partial x_j=2\partial q/\partial x_j^\top G^{-1} q -q^\top G^{-1}B_jG^{-1}q$.
A: $
\def\l{\left}
\def\r{\right}
\def\o{{\tt1}}
\def\p{{\partial}}
\def\g#1#2{\frac{\p #1}{\p #2}}
\def\c#1{\color{red}{#1}}
\def\B{{\mathbb B}}
\def\R{{\mathbb R}^p}
\def\E{{\cal E}}
$Let $\{e_k\}$ denote the standard basis vectors for $\R$ and $M$ the gradient of $q,\;$ then
$$M = \g{q}{x} \quad\implies\quad dq = M\,dx$$
The gradient of a matrix $(G)$ with respect to a vector $(x)$ is a third-order tensor $(\B)$ with components
$$\eqalign{
\B_{ijk} &= \g{G_{ij}}{x_k} \\
}$$
Using this tensor we can write the other terms in the problem as
$$\eqalign{
B_k &= \B\cdot e_k \;\;\implies\;\; \B = \sum_{k=1}^p B_k\star e_k \\
\B\cdot x &= \sum_{k=1}^p B_k\star\l(e_k\cdot x\r) = \sum_{k=1}^p B_k x_k \\
G &= A + \B\cdot x \\
}$$
where $(\star)$ denotes the tensor/dyadic product,
and $(\cdot)$ the dot standard product.
Let's also define a new vector $(w)$ with components
$$\eqalign{
w_k = \sum_{k=1}^p G^{-1}qq^TG^{-1}:B_k \\
}$$
where $(:)$ denotes the double-dot product and is an alternative
notation for the trace, i.e.
$$\eqalign{
A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij}
 \;=\; {\rm Tr}\!\l(AB^T\r) \\
A:A &= \big\|A\big\|_F^2 \\
}$$
Now we can write the function and calculate its differential and gradient.
$$\eqalign{
f &= q^TG^{-1}q \\
  &= G^{-1}:qq^T \\
df&= G^{-1}:\l(dq\,q^T+q\,dq^T\r) + qq^T:dG^{-1} \\
  &= \l(G^{-1}+G^{-T}\r):dq\,q^T - qq^T:G^{-1}dG\,G^{-1} \\
  &= 2G^{-1}:dq\,q^T - G^{-1}qq^TG^{-1}:dG \\
  &= 2G^{-1}q:M\,dx - G^{-1}qq^TG^{-1}:\B\cdot dx \\
  &= \l(2MG^{-1}q - G^{-1}qq^TG^{-1}:\B\r)\cdot dx \\
  &= \l(2MG^{-1}q - w\r)\cdot dx \\
\g{f}{x} &= 2MG^{-1}q - w \\
}$$
