# Is there any way to calculate change in derivatives along a vector?

Suppose I have a function F($\vec{x})$ $x\subset R^{n}$, and a vector $\vec{g}$. I want to perturb $\vec{x}$ along the $\vec{g}$ vector, and see how the gradient changes as I move further along the $\vec{g}$ vector. So I'm trying to solve $\lim_{h\to 0}(\nabla F(\vec{x}+h*\vec{g})-\nabla F(\vec{x}))/h$. Note, I don't want to know the change in the function as h approaches 0, but the change in each gradient (so there should be as many terms in the answer to the above formula as there are dimensions in $\vec{x}$. I'm pretty sure there should be an efficient way to do this (linear in the number of parameters), since at worst case I can just use a small value for h and use finite differences. Likewise, I think we should be able to get higher derivatives along the vector also linear in the number of parameters, since we can also accomplish that linear in the number of parameters with finite differences.

My first thought was to set $\vec{x}(h)=\vec{x}_{o}+h*\vec{g}$, and then try to use the chain rule, so I end up with $F'(\vec{x}(h))*\vec{x}'(h)*dh$, but that quantity seems like it would be one dimensional ($\frac{dF}{dh}$), and not the change in the each gradient itself. Plus, I'm not sure that would even work for higher derivatives, since $F''(\vec{x})$ has $n^{2}$ terms, so we're no longer linear in the number of parameters.

Now I'm trying to see if I can derive some transformation of F that will change it to a new function that has as its derivatives the quantities I'm trying to calculate, but I'm not having much luck doing it that way either. In general, I'd like to be able to calculate the change in the gradients along h even if $\vec{x}(h)$ is a non-linear function of h (so we're following a curve instead of a line), but for the time being I'm keeping it simple and saying that $\vec{x}(h)$ is linear.

Is there some general way to find these quantities.

-

If I understand correctly, you are trying to take the directional derivative of the gradient of $F$ along $g$. $(\nabla F)^T$ is just another (vector-valued) function of $x$, and its directional derivative along $g$ is the dot product of the gradient of $(\nabla F)^T$ with g, ie, $$HF g,$$ where $$HF = \nabla (\nabla F)^T = \left[\begin{array}{ccc}\frac{\partial^2 F}{\partial x_1^2} & \frac{\partial^2 F}{\partial x_1 \partial x_2} & \ldots\\ \frac{\partial^2 F}{\partial x_1 \partial x_2} & \frac{\partial^2F}{\partial x_2^2} & \ldots\\\vdots & \vdots & \ddots \end{array}\right]$$ is the Hessian of $F$.
Unfortunately, computing this matrix-vector product for general $F$ is roughly cubic in the dimension $n$, and not linear. In many applications, however, $HF$ is sparse, so that computing and using it remains preferable to taking finite differences of $\nabla f$ due to the former's greater accuracy.