Help computing derivative of a function Let $x, y$ be real vectors in $\mathbb{R}^n$; we denote the usual inner product with $(x, y) \mapsto x^Ty$.
Let $P, Q$ be $n \times n$ real, positive definite matrices. We consider the function
$$
f(t) = (tx + y)^T(P + tQ)^{-1}(tx + y).
$$
This is a function $\mathbb{R} \to \mathbb{R}$. When $n = 1$, the derivative is easily computed as
$$
f'(t) = \frac{-(tx + y)(yq - x(tq + 2p))}{(p + tq)^2}.
$$
Is there a formula for the general case $n > 1$? I suppose it requires understanding the derivative of the mapping $t \mapsto (P + tQ)^{-1}$.
 A: Yes. Note that we may write
$$
g(t) = tx + y, \quad \mbox{and} \quad h(t) = (P+tQ)^{-1} (tx + y). 
$$
Clearly we have $g'(t) = x$. Additionally,
$$
h'(t) = (P + tQ)^{-1}x - (P+tQ)^{-1}Q(P+tQ)^{-1}(tx + y).
$$
The second term above follows from the identity $A(t) A(t)^{-1} = I$.
Putting the pieces together:
$$
f'(t) = g(t)^Th'(t) + h(t)^Tg'(t) = 2x^T(P+ tQ)^{-1}(tx + y) - (tx + y)(P+tQ)^{-1}Q(P+tQ)^{-1}(tx + y).
$$
Note that in the one-dimnesional case, this gives
$$
f'(t) = -\frac{(tx+y)(qtx + qy - 2x(tq + p))}{(p + tq)^2} = 
-\frac{(tx+y)(qy - x(tq + 2p))}{(p + tq)^2},
$$
which was our original formula.
A: The objective function writes
$f = \mathbf{u}
:
\mathbf{A}^{-1} \mathbf{u}
$
where
$\mathbf{u}=t\mathbf{x} + \mathbf{y},
\mathbf{A}=\mathbf{P} + t\mathbf{Q}$
and
the colon operator means the Frobenius
inner product.
It holds
\begin{eqnarray*}
df
&=& 
(
\mathbf{A}^{-1}+\mathbf{A}^{-T}
)
\mathbf{u}:
d\mathbf{u} +
\mathbf{u}\mathbf{u}^T
:d\mathbf{A}^{-1} \\
&=&
(
\mathbf{A}^{-1}+\mathbf{A}^{-T}
)
\mathbf{u}:
\mathbf{x}dt -
\mathbf{A}^{-T}
\mathbf{u}\mathbf{u}^T
\mathbf{A}^{-T}
:
d\mathbf{A} \\
&=&
\left[
(
\mathbf{A}^{-1}+\mathbf{A}^{-T}
)
\mathbf{u}:
\mathbf{x}
-
\mathbf{A}^{-T}
\mathbf{u}\mathbf{u}^T
\mathbf{A}^{-T}
:
\mathbf{Q}
\right]
dt
\end{eqnarray*}
The derivative
is
\begin{eqnarray*}
\frac{\partial f}{\partial t}
&=&
\mathbf{x}^T
(\mathbf{A}^{-1}+\mathbf{A}^{-T})
\mathbf{u}
-
\mathrm{tr}
\left(
\mathbf{A}^{-1}
\mathbf{u}\mathbf{u}^T
\mathbf{A}^{-1}
\mathbf{Q}
\right) \\
&=&
\mathbf{x}^T
(\mathbf{A}^{-1}+\mathbf{A}^{-T})
\mathbf{u}
-
\mathbf{u}^T 
\mathbf{A}^{-1} \mathbf{Q} \mathbf{A}^{-1}
\mathbf{u}
\end{eqnarray*}
If $\mathbf{A}$ is not symmetric,
this is the correct formula for the derivative.
