# implicit differentiation when the implicit equation is a differential equation

I want to find derivative/variation (I think variation is more correct) of solution of a differential equation with respect to one parameter of differential equation but without solving the differential equation explicitly.

e.g. consider this differential equation $X'(t)=A X(t) + B(t)$, I need to find the derivative of solution with respect to A (assume a general parametric initial condition).

Actually the problem arise from a constrained optimization problem where the differential equation is the constraint and the objective function is an explicit function of solution of differential equation and so implicit function of the differential equation parameters and I need to find the optimum parameters.

The actual differential equation is wave PDE which is discretized by some method like finite difference (the second order wave PDE is considered as 2 simultaneous first order equation and after discretization 2 sets of equation stacked together), so the coefficient matrix or parameters are properties of medium where the wave propagates (local propagation speed of wave).

Considering solving the optimization problem is the main goal, and the problem must be solved numerically, any approximation of this derivative in order to be used in a gradient descent algorithm or any other iterative solution is welcomed.

• Is $X(t)$ a vector or a matrix? – Saad Jun 6 '18 at 7:21
• @AlexFrancisco, X and b are vectors and A is a matrix , but i think if there were a general method for single scalar equation i could expand it to vector case. – Mohammad M Jun 6 '18 at 7:24
• It seems that $X(t)$ still has to be solved first since for any $a_{i,j}$ in $A\in M_{n×n}(\mathbb R)$, $Y=\dfrac{\partial X}{\partial a_{i,j}}$ satisfies the equation$$Y'(t)=AY(t)+E_{i,j}X(t),$$where $E_{i,j}\in M_{n×n}(\mathbb R)$ has $1$ as its $(i,j)$-th entry and $0$ as other entries. – Saad Jun 6 '18 at 7:57

## 3 Answers

Let $Y$ be the solution to $Y'=(A+H)Y+b$, then $$Y'-X'=(A+H)(Y-X)+HX$$ so that in first order for $H=\epsilon\Delta A$ and $Y=X+ϵΔX$ we get for the directional derivative $$ΔX'=A\,ΔX + HX$$ This has an explicit solution formula, but there will be terms like $\int e^{-sA}He^{sA}b(s)\,ds$ involved where you can not extract $H$ as it does in general not commute with $A$.

• Thank you. I have edited my question after your answer. my main problem is to solve a constrained optimization problem. I was hoping to use a gradient descent algorithm after obtaining this derivative which seems to be impossible. – Mohammad M Jun 6 '18 at 10:10

Let $$X'(t)=AX(t)+B(t),\quad X(t_0) = X_0,$$ $$\widehat X'(t)=\widehat A\widehat X(t)+B(t),\quad \widehat X(t_0) = X_0,$$ $$\bar X(t) = \widehat X(t) - X(t),$$ $$\bar A(t) = \widehat A(t) - A(t),$$ then $$\left(e^{-At}X(t)\right)' = e^{-At}B(t),$$ $$\left(e^{-\widehat At}\widehat X(t)\right)' = e^{-\widehat At}B(t),$$ $$\left(e^{-\widehat At}\widehat X(t) - e^{-At}X(t)\right)' = \left(e^{-\widehat At} - e^{-At}\right)B(t),$$ $$\left(e^{-\widehat At}\widehat X(t) - e^{-At}X(t)\right)\Bigg|_{t_0}^t = \int\limits_{t_0}^t\left(e^{-\widehat At} - e^{-At}\right)B(t)dt,$$ $$e^{-\widehat At}\widehat X(t) - e^{-At}X(t) = \left(e^{-\widehat At_0} - e^{-At_0}\right)X_0 + \int\limits_{t_0}^t\left(e^{-\widehat At} - e^{-At}\right)B(t)dt,$$ $$\boxed{\bar X(t) = \left(e^{\bar At}-1\right)X(t) + \left(e^{\bar A(t-t_0)} - e^{\bar At}\right)e^{A(t-t_0)}X_0 + e^{\widehat At}\int\limits_{t_0}^t\left(e^{-\widehat At} - e^{-At}\right)B(t)dt}.$$

If we multiply $X'(t)=AX(t)+B(t)$ by $\mathrm{e}^{-tA}$ we obtain $$\mathrm{e}^{-tA}\big(X'(t)-AX(t)\big)=\mathrm{e}^{-tA}B(t)$$ or $$\big(\mathrm{e}^{-tA}X(t)\big)'=\mathrm{e}^{-tA}B(t),$$ and hence, fixing a $\tau$ in the domain of $B(t)$, then $$\mathrm{e}^{-tA}X(t)-\mathrm{e}^{-\tau A}X(\tau)=\int_\tau^t \mathrm{e}^{-sA}B(s)\,ds$$ or equivalently $$X(t)=X(t;A)=\mathrm{e}^{(t-\tau)A}X(\tau)+\int_\tau^t \mathrm{e}^{(t-s)A}B(s)\,ds.$$ If $A=(a_{ij})$, then $$\frac{\partial}{\partial a_{ij}}X(t;A)=\frac{\partial}{\partial a_{ij}}\mathrm{e}^{(t-\tau)A}X(\tau)+\int_\tau^t \frac{\partial}{\partial a_{ij}}\mathrm{e}^{(t-s)A}B(s)\,ds$$

• Thank you. Using propagation operator is some sort of finding explicit solution. The problem is the calculation of derivative of this operator with respect to elements of A matrix which seems to be impossible to give a closed form (considering the Taylor's expansion of exponential function and taking derivatives with respect to A's elements) – Mohammad M Jun 6 '18 at 9:59