0
$\begingroup$

I am looking to solve an optimization problem, using a gradient based method. As such, I need to calculate the gradient of the objective function. The objective function, however, contains an ODE that's solved using Runge-Kutta 4th order. So I am wondering how I would analytically compute the gradient, without using finite difference.

Here's a simple example to illustrate that problem I am having. Let's assume I have an objective function that contains an ODE $$ \frac{dy}{dt} = \frac{k t^2-y}{e^{y+t}} $$ for which k is the design variable, and the objective function value is y(t=1), solved using RK4.

My problem is I am unsure of how I can solve for the gradient of the objective with respect to the design variable, namely $$ \frac{dy}{dk}(t=1) $$ I would like to solve for this analytically rather than relying on finite difference of any order.

$\endgroup$
  • 1
    $\begingroup$ Usually you would just compute an explicit ODE for $\frac{d}{dt} \frac{dy}{dk}$ and solve that using the same method as you used to solve the main ODE. $\endgroup$ – Ian Jan 17 '18 at 1:38
0
$\begingroup$

Set $u=\frac{d y}{d k}$, then $u(0)=0$ if there is no previous dependence on $k$ and $$ \frac{d}{dt}u(t)=\frac{d}{dk}\frac{d}{dt}y=\frac{d}{dk}\frac{kt^2−y}{e^{y+t}} =\frac{t^2-u-(kt^2−y)u}{e^{y+t}} $$ so that you can compute the target value $u(1)$ by solving the coupled system.

$\endgroup$
  • $\begingroup$ Ah ok, thanks for the suggestion. Should this give an exact gradient when both the original ODE and the gradient is solved using the same step size? I am comparing the gradient I get vs finite difference (FD) as well as complex step (CS), CS and FD matches to 9 decimal places, while this solution only matches to 2 decimal place. $\endgroup$ – E.L. Jan 18 '18 at 4:10
  • $\begingroup$ Without seeing your code, I can't say what went wrong. If the derivative is integrated simultaneously with $y$, that is you integrate the vector $[y,u]$, then I would expect that the derivative is as correct as the solution itself. $\endgroup$ – LutzL Jan 18 '18 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.