3
$\begingroup$

Given is the function $f : \mathbb{R}^p \to \mathbb{R}$ with

$$ f(x) = q(x)^{\top} G^{-1} q(x) $$

where $G = A + x_1 B_1 + \ldots + x_p B_p$.

  • The matrices $A, B_1, \ldots, B_p \in \mathbb{R}^{n \times n}$ are all symmetric positive definit.

  • $q: \mathbb{R}^n \to \mathbb{R}^n$ and the Jacobian $\nabla q$ is known.

Is it possible to derive a closed form for $\nabla f$? For me, the hard part is $G^{-1}$. Any hints or suggestions are really appreciated!

$\endgroup$
1
  • $\begingroup$ You should end up with the solution $\nabla f = 2 \nabla q G^{-1} q$ $\endgroup$ Nov 26 '20 at 23:45
3
$\begingroup$

$$\frac1{\Delta x}\big[(A+(x+\Delta x)B)^{-1}-(A+xB)^{-1}\big]=$$ $$\frac1{\Delta x}(A+xB)^{-1}(A+xB)\big[(A+(x+\Delta x)B)^{-1}-(A+xB)^{-1}\big] (A+(x+\Delta x)B) (A+(x+\Delta x)B)^{-1}$$ $$=-(A+xB)^{-1}B (A+(x+\Delta x)B)^{-1}\rightarrow-(A+xB)^{-1}B(A+xB)^{-1}$$ as $\Delta x\rightarrow0$.

So that $\partial f/\partial x_j=2\partial q/\partial x_j^\top G^{-1} q -q^\top G^{-1}B_jG^{-1}q$.

$\endgroup$
2
$\begingroup$

$ \def\l{\left} \def\r{\right} \def\o{{\tt1}} \def\p{{\partial}} \def\g#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}} \def\B{{\mathbb B}} \def\R{{\mathbb R}^p} \def\E{{\cal E}} $Let $\{e_k\}$ denote the standard basis vectors for $\R$ and $M$ the gradient of $q,\;$ then $$M = \g{q}{x} \quad\implies\quad dq = M\,dx$$ The gradient of a matrix $(G)$ with respect to a vector $(x)$ is a third-order tensor $(\B)$ with components $$\eqalign{ \B_{ijk} &= \g{G_{ij}}{x_k} \\ }$$ Using this tensor we can write the other terms in the problem as $$\eqalign{ B_k &= \B\cdot e_k \;\;\implies\;\; \B = \sum_{k=1}^p B_k\star e_k \\ \B\cdot x &= \sum_{k=1}^p B_k\star\l(e_k\cdot x\r) = \sum_{k=1}^p B_k x_k \\ G &= A + \B\cdot x \\ }$$ where $(\star)$ denotes the tensor/dyadic product, and $(\cdot)$ the dot standard product.

Let's also define a new vector $(w)$ with components $$\eqalign{ w_k = \sum_{k=1}^p G^{-1}qq^TG^{-1}:B_k \\ }$$ where $(:)$ denotes the double-dot product and is an alternative notation for the trace, i.e. $$\eqalign{ A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; {\rm Tr}\!\l(AB^T\r) \\ A:A &= \big\|A\big\|_F^2 \\ }$$ Now we can write the function and calculate its differential and gradient. $$\eqalign{ f &= q^TG^{-1}q \\ &= G^{-1}:qq^T \\ df&= G^{-1}:\l(dq\,q^T+q\,dq^T\r) + qq^T:dG^{-1} \\ &= \l(G^{-1}+G^{-T}\r):dq\,q^T - qq^T:G^{-1}dG\,G^{-1} \\ &= 2G^{-1}:dq\,q^T - G^{-1}qq^TG^{-1}:dG \\ &= 2G^{-1}q:M\,dx - G^{-1}qq^TG^{-1}:\B\cdot dx \\ &= \l(2MG^{-1}q - G^{-1}qq^TG^{-1}:\B\r)\cdot dx \\ &= \l(2MG^{-1}q - w\r)\cdot dx \\ \g{f}{x} &= 2MG^{-1}q - w \\ }$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.