I am dealing with a parameter identification problem, where I have a system of differential equations $$\frac{dy}{dt}=\mathbf{F}(t,\vec{y}(t,\vec{p}),\vec{p}) \tag1$$ where I need to find the parameters contained in the vector $\vec{p}$, that minimise the objective function $$C=\min_p\left(||\vec{y}(t,\vec{p})-\vec{y}_m||^2_2\right) \tag2$$ where $\vec{y}_m$ is some measured data.

As I am performing the optimisation using a gradient-based algorithm, I need to find the gradient of the objective function. For this as I understand I need to employ the sensitivity (or variational) equations where (for notational simplicity $\vec{\cdot}$ is ommitted): $$\frac{d}{dt}\frac{\partial y}{\partial p}=\frac{\partial \mathbf{F}}{\partial y}\cdot\frac{\partial y}{\partial p}+\frac{\partial \mathbf{F}}{\partial p} \tag3$$

This way the gradient of my objective function would be: $$\vec{G}=\left[\frac{\partial y}{\partial p}\right]^T\cdot\left(y-y_m\right)$$

There are three questions I have:

  1. Is the expression for the gradient $G$ in equation (4) correct?
  2. How is equation (3) derived? This post seems to do it with the chain rule, however Hemker (1971) on page 75 seems to be performing a linear approximation. These seem fairly different, with different assumptions made particularly in Hemker's work, which also seems to result to a different form of the objective function's gradient (which I do not comprehend really well).
  3. For my case I have the differential equations in the form of

$$\mathbf{ M(\mathit{y})\mathit{y''} +C\mathit{y'}+K\mathit{y}+ A(\mathit{y,y',p})} = \mathbf{0}\\ \Rightarrow \mathbf{F} = \begin{bmatrix} y' \\y'' \end{bmatrix}=\begin{bmatrix} \mathbf{0} & \mathbf{I} \\ \mathbf{-[M(\mathit{y})]^{-1}K} & \mathbf{-[M(\mathit{y})]^{-1}C} \end{bmatrix}\begin{bmatrix} y \\ y' \end{bmatrix}+ \\ \begin{bmatrix} \mathbf{0} \\ \mathbf{-[M(\mathit{y})]^{-1}A(\mathit{y,y',p})} \end{bmatrix} \tag4$$

where $\mathbf{M(\mathit{y})}$ is a diagonal variable matrix, $\mathbf{A(\mathit{y,y',p})}$ is a variable vector, and $\mathbf{C,K}$ are constant tri-diagonal matrices. So $$\begin{align} \frac{\partial \mathbf{F}}{\partial y} = & \begin{bmatrix} \mathbf{0} & \mathbf{0} \\ [\mathbf{M(\mathit{y})}]^{-1}\frac{\partial M}{\partial y}[\mathbf{M(\mathit{y})}]^{-1}(\mathbf{K\mathit{y}+ C\mathit{y'}})-\mathbf{[M(\mathit{y})]^{-1}K} & \mathbf{0} \end{bmatrix}+\\ & \begin{bmatrix} \mathbf{0} & \mathbf{0} \\ [\mathbf{M(\mathit{y})}]^{-1}\frac{\partial M}{\partial y}[\mathbf{M(\mathit{y})}]^{-1}\text{diag}\mathbf{\left(A(\mathit{y,y',p})\right)-[\mathbf{M(\mathit{y})}]^{-1}}\frac{\mathbf{\partial A(\mathit{y,y',p})}}{\partial y} & \mathbf{0} \end{bmatrix} \end{align} \tag5$$

However when employing equation (5) to solve equation (3) and subsequently find $G$, this does not give rise to the gradient of the objective function (I know since I have the correct answer which is numerically derived, and my $C$ is very smooth). Instead what returns an answer extremely close to the correct one (and I can't explain that) is if I use the following:

$$\begin{align} \frac{\partial \mathbf{F}}{\partial y} = & \begin{bmatrix} \mathbf{0} & \mathbf{0} \\ -[\mathbf{M(\mathit{y})}]^{-1}\frac{\partial M}{\partial y}[\mathbf{M(\mathit{y})}]^{-1}(\mathbf{K\mathit{y}+ C\mathit{y'}})-\mathbf{[M(\mathit{y})]^{-1}K} & \mathbf{0} \end{bmatrix}-\\ & \begin{bmatrix} \mathbf{0} & \mathbf{0} \\ [\mathbf{M(\mathit{y})}]^{-1}\frac{\partial M}{\partial y}[\mathbf{M(\mathit{y})}]^{-1}\text{diag}\mathbf{\left(A(\mathit{y,y',p})\right)} & \mathbf{0} \end{bmatrix} \end{align} \tag6$$

So to conclude with my 3rd question, based on the above, am I missing part of the theory here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.