1
$\begingroup$

I have an algorithm that produces a set of real outputs given real inputs. For practical purposes, let's say I have two inputs and one output, and the algorithm can be represented by the function $\phi: \Re² \rightarrow \Re$.

I need to calculate $\frac{\partial^2 \phi(u_1,u_2)}{\partial u_1\partial u_2} \Bigr|_{\bar{u_1}\bar{u_2}}$ but I don't have an explicit formula for $\phi$. My only option is to run the algorithm to obtain the output given the input values.

My original idea was to run the program with a series of values around $\bar{u_1}$ and $\bar{u_2}$, then calculate $\frac{\partial \phi}{\partial u_1}$ at the different fixed $u_2$ values, and finally taking $\frac{\partial }{\partial u_2}$ to that (i.e. taking three values for $u_1$ and three for $u_2$ this would be running the algorithm for all the combinations $\{\bar{u_1} - \delta, \bar{u_1} , \bar{u_1} + \delta\} \times \{\bar{u_2} - \delta, \bar{u_2} , \bar{u_2} + \delta\}$, computing the derivative of $\phi$ with respect to $u_1$ for each of the three fixed $u_2$ values, and finally taking the derivative of that with respect to $u_2$ .)

However I notice that the calculated derivate depends heavily on the spacing between the values of the inputs $u_1$ and $u_2$ (i.e. $\delta$.) Moreover, I don't see it converging as I make the spacing smaller.

Is this a numerical problem or is it that the function is simply not differentiable? How can I tell?

In the following picture there's the resulting plot for $\delta = 0.01$ (left) and $\delta = 0.02$ (right). The derivative was calculated in $\texttt{R}$ using the $\texttt{splines}$ package.

enter image description here

EDIT: If I plot $\phi$ with 21 values for each independent variable I get,

With $\delta = 0.05$

enter image description here

With $\delta = 0.001$

enter image description here

$\endgroup$
  • 1
    $\begingroup$ There is an obvious problem with the evaluation of the $\phi$ function. Try reducing the steps to see if you get anything smoother. If not, question the algorithm. If the algorithm is out of the picture, drop the idea of derivating the wild function at all. $\endgroup$ – Yves Daoust Feb 28 '17 at 13:43
2
$\begingroup$

In general, you can't tell. More specifically, one could formally prove that there is no algorithm that will correctly answer such a question for all $\phi$ (representable as outputs of algorithms).

In practice, it does not matter: if your function is only differentiable in a tiny, tiny region around the point of interest, so tiny that you start having numerical stability problems when trying to get into it, that differentiability is useless to you -- it's as if it were not there.

$\endgroup$
1
$\begingroup$

Just an idea.

Why not to try to compute nine points on a square, the central point being the one for which you want the second derivative (this is what you propose) and now write $$f(x,y)=a+bx+cy+dx^2+ey^2+gxy$$ and use matrix calculation just as for a multilinear regression.

I suppose that this would be more stable than computing derivatives of derivatives and the order of operations will not affect the result.

$\endgroup$
  • $\begingroup$ Wouldn't I still have the same problem with the size of $\delta$? $\endgroup$ – milo Feb 28 '17 at 11:09
  • $\begingroup$ @milo. By the way, are you aware of $svcm(.)$ in $R$ ? It does multidimension splines. As I said, I do not enjoy the idea of computing derivatives of derivatives. $\endgroup$ – Claude Leibovici Feb 28 '17 at 11:15
  • $\begingroup$ @milo. Have a look at holoborodko.com/pavel/numerical-methods/numerical-derivative/… there are very interesting things for your needs. $\endgroup$ – Claude Leibovici Feb 28 '17 at 11:27
  • $\begingroup$ I added the plot of $\phi$ to the question. Would a polynomial fit help in this case? $\endgroup$ – milo Feb 28 '17 at 13:34
  • $\begingroup$ After looking at the plot you added, I am crying ! If there are such apparent discontinuities, I am afraid by the problem. Did you try the methods given in the link I gave earlier ? In any manner, the step size need to be small (say $\Delta x=0.001\times x$). $\endgroup$ – Claude Leibovici Feb 28 '17 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.