implicit differentiation when the implicit equation is a differential equation I want to find derivative/variation (I think variation is more correct) of solution of a differential equation with respect to one parameter of differential equation but without solving the differential equation explicitly.
e.g. consider this differential equation $X'(t)=A X(t) + B(t)$, I need to find the derivative of solution with respect to A (assume a general parametric initial condition). 
Actually the problem arise from a constrained optimization problem where the differential equation is the constraint and the objective function is an explicit function of solution of differential equation and so implicit function of the differential equation parameters and I need to find the optimum parameters.
The actual differential equation is wave PDE which is discretized by some method like finite difference (the second order wave PDE is considered as 2 simultaneous first order equation and after discretization 2 sets of equation stacked together), so the coefficient matrix or parameters are properties of medium where the wave propagates (local propagation speed of wave).
Considering solving the optimization problem is the main goal, and the problem must be solved numerically, any approximation of this derivative in order to be used in a gradient descent algorithm or any other iterative solution is welcomed.
 A: Let $Y$ be the solution to $Y'=(A+H)Y+b$, then
$$
Y'-X'=(A+H)(Y-X)+HX
$$
so that in first order for $H=\epsilon\Delta A$ and $Y=X+ϵΔX$ we get for the directional derivative
$$
ΔX'=A\,ΔX + HX
$$
This has an explicit solution formula, but there will be terms like $\int e^{-sA}He^{sA}b(s)\,ds$ involved where you can not extract $H$ as it does in general not commute with $A$.
A: Let
$$X'(t)=AX(t)+B(t),\quad X(t_0) = X_0,$$
$$\widehat X'(t)=\widehat A\widehat X(t)+B(t),\quad \widehat X(t_0) = X_0,$$
$$\bar X(t) = \widehat X(t) - X(t),$$
$$\bar A(t) = \widehat A(t) - A(t),$$
then
$$\left(e^{-At}X(t)\right)' = e^{-At}B(t),$$
$$\left(e^{-\widehat At}\widehat X(t)\right)' = e^{-\widehat At}B(t),$$
$$\left(e^{-\widehat At}\widehat X(t) - e^{-At}X(t)\right)' = \left(e^{-\widehat At} - e^{-At}\right)B(t),$$
$$\left(e^{-\widehat At}\widehat X(t) - e^{-At}X(t)\right)\Bigg|_{t_0}^t =  \int\limits_{t_0}^t\left(e^{-\widehat At} - e^{-At}\right)B(t)dt,$$
$$e^{-\widehat At}\widehat X(t) - e^{-At}X(t) = \left(e^{-\widehat At_0} - e^{-At_0}\right)X_0 +  \int\limits_{t_0}^t\left(e^{-\widehat At} - e^{-At}\right)B(t)dt,$$
$$\boxed{\bar X(t) = \left(e^{\bar At}-1\right)X(t) + \left(e^{\bar A(t-t_0)} - e^{\bar At}\right)e^{A(t-t_0)}X_0 + e^{\widehat At}\int\limits_{t_0}^t\left(e^{-\widehat At} - e^{-At}\right)B(t)dt}.$$
A: If we multiply $X'(t)=AX(t)+B(t)$ by $\mathrm{e}^{-tA}$ we obtain
$$
\mathrm{e}^{-tA}\big(X'(t)-AX(t)\big)=\mathrm{e}^{-tA}B(t)
$$
or
$$
\big(\mathrm{e}^{-tA}X(t)\big)'=\mathrm{e}^{-tA}B(t),
$$
and hence, fixing a $\tau$ in the domain of $B(t)$, then
$$
\mathrm{e}^{-tA}X(t)-\mathrm{e}^{-\tau A}X(\tau)=\int_\tau^t \mathrm{e}^{-sA}B(s)\,ds
$$
or equivalently
$$
X(t)=X(t;A)=\mathrm{e}^{(t-\tau)A}X(\tau)+\int_\tau^t \mathrm{e}^{(t-s)A}B(s)\,ds.
$$
If $A=(a_{ij})$, then
$$
\frac{\partial}{\partial a_{ij}}X(t;A)=\frac{\partial}{\partial a_{ij}}\mathrm{e}^{(t-\tau)A}X(\tau)+\int_\tau^t \frac{\partial}{\partial a_{ij}}\mathrm{e}^{(t-s)A}B(s)\,ds
$$
