Suppose we are given a vector field $\vec{a}$ such that

$$\vec{a}(x_1,\ldots,x_n)=\sum_{i=1}^{k}f_i(x_1,\ldots,x_n)\vec{e_i} $$


$$\mathbf{S}=\{\vec{e_1},\ldots,\vec{e_k}\}$$ is some constant, orthonormal basis of $\Bbb{R}^k$.

What follows is to be taken with a cellar of salt. To compute the directional derivative, we start with the gradient. Its components are given by the matrix $\mathbf{G}$:

$$\mathbf{G}=\begin{bmatrix}\frac{\partial f_1(x_1,\ldots,x_n)}{\partial x_1} & \cdots &\frac{\partial f_1(x_1,\ldots,x_n)}{\partial x_n}\\ \vdots & \ddots & \vdots\\\frac{\partial f_k(x_1,\ldots,x_n)}{\partial x_1}&\cdots&\frac{\partial f_k(x_1,\ldots,x_n)}{\partial x_n}\end{bmatrix}.$$

The gradient $\vec{\nabla}\vec{a}$ itself is given by the double sum

$$\vec{\nabla}\vec{a}=\sum_{i=1}^{k}\sum_{j=1}^{n}\frac{\partial f_i(x_1,\ldots,x_n)}{\partial x_j}\vec{e_i}\otimes\vec{e_j}.$$ When dealing with scalar-valued functions, the derivative in the direction of some vector $\vec{u}$ would be the projection of the gradient onto $\vec{u}$.

Assuming this still holds, the directional derivative $\mathrm{D}_{\vec{u}}(\vec{a})$ of $\vec{a}$ is


Substituting in our double sum:

$$\mathrm{D}_{\vec{u}}(\vec{a})=\frac{\vec{u}}{|\vec{u}|}\sum_{i=1}^{k}\sum_{j=1}^{n}\frac{\partial f_i(x_1,\ldots,x_n)}{\partial x_j}\vec{e_i}\otimes\vec{e_j}.$$

Question: Is this generalisation for $\mathrm{D}_{\vec{u}}(\vec{a})$ true?

  • If so, how does one evaluate it?
  • If not, what is the proper way to find a directional derivative of a vector field?


The sign $\otimes$ denotes the tensor product. Here, we have the tensor product of basis vectors.

Furthermore, following dyadics on Wikipidia, it seems for an orthonormal basis $$\mathrm{D}_{\vec{u}}(\vec{a})=\frac{\vec{u}}{|\vec{u}|}\mathbf{G}.$$ So if $\vec{u}=\vec{e_m}$, then $$\mathrm{D}_{\vec{e_m}}(\vec{a})=\vec{e_m}\mathbf{G}.$$ This makes no sense, unless it is some kind of tensor contraction... In such a case, $$\mathrm{D}_{\vec{e_m}}(\vec{a})=\begin{bmatrix}\sum_{i=1}^{k}e_iG_{i1}\\ \vdots \\ \sum_{i=1}^{k}e_iG_{in}\end{bmatrix}.$$

Here $e_i$ denotes the $i^{th}$ component of $\vec{e_m}$; $G_{ij}$ denotes the $ij^{th}$ component of $\mathbf{G}$. And since we are in an orthonormal basis, only $e_m=1\neq0$:

$$\mathrm{D}_{\vec{e_m}}(\vec{a})=\begin{bmatrix}e_mG_{m1}\\ \vdots \\ e_mG_{mn}\end{bmatrix}=\begin{bmatrix}G_{m1}\\ \vdots \\ G_{mn}\end{bmatrix}.$$

This seems to be the $m^{th}$ row of $\mathbf{G}$ transposed. And in derivative form,

$$\mathrm{D}_{\vec{e_m}}(\vec{a})=\begin{bmatrix}\frac{\partial f_m(x_1,\ldots,x_n)}{\partial x_1}\\ \vdots \\ \frac{\partial f_m(x_1,\ldots,x_n)}{\partial x_n}\end{bmatrix}.$$

  • $\begingroup$ en.wikipedia.org/wiki/Lie_derivative $\endgroup$
    – user8960
    Oct 19, 2016 at 18:56
  • $\begingroup$ @user8960: Cognisant of the possibility of seeming ignorant... Is that to say the formula I gave is not true, and the correct approach would be the Lie derivative (LD)? Or is the LD in this case equivalent to calculating as postulated? And the LD is a further generalisation for p-order tensor fields? Also, I have the non-rigorous feeling that the equation for $$\mathrm{D}_{\vec{u}}(\vec{a})$$ simplifies quite a bit if $\vec{u}$ is one of the vectors $\vec{e_i}$. Is this true? $\endgroup$ Oct 19, 2016 at 19:33
  • $\begingroup$ Something else to be sure of: make sure your basis vectors, $\hat{e}_i$, are position independent, otherwise their derivatives will have non-trivial contributions. $\endgroup$
    – Sean Lake
    Oct 19, 2016 at 21:24
  • $\begingroup$ @SeanLake: duly noted. Will edit thread. $\endgroup$ Oct 19, 2016 at 21:29
  • $\begingroup$ Isn't the directional derivative just the product of the Jacobian matrix and the direction vector? $\endgroup$ Mar 2, 2018 at 21:00

2 Answers 2


Suppose the vector-valued function $\mathbf{f}:\mathbb{R}^n\rightarrow\mathbb{R}^m$ has the (total) derivative at $\mathbf{x_0}\in \mathbb{R}^n$ denoted by $\mathrm{d}_\mathbf{x_0}\mathbf{f}$. It is a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$. It gives the (total) differential of the function $\mathbf{f}$ at $\mathbf{x_0}$ as a function mapping from $\mathbb{R}^n$ to $\mathbb{R}^m$ by applying to the vector variable $\mathbf{x}$ near $\mathbf{x_0}$ to give $\mathrm{d}_\mathbf{x_0}\mathbf{f}\left(\mathbf{x}-\mathbf{x_0}\right)$. With respect to standard basis sets $\left\{\mathbf{\hat{a}}_i\right\}_{i=1}^{n}$ and $\left\{\mathbf{\hat{b}}_i\right\}_{i=1}^{m}$ of $\mathbb{R}^n$ and $\mathbb{R}^m$, respectively, the total derivative $\mathrm{d}_\mathbf{x_0}\mathbf{f}$ corresponds to the $ m \times n$ matrix called the Jacobian matrix

$$\left(\mathrm{d}_\mathbf{x_0}\mathbf{f}\right)=\left(\begin{matrix}\frac{\partial f_1}{\partial x_1}&&\cdots&&\frac{\partial f_1}{\partial x_n}\\\vdots&&\ddots&&\vdots\\\frac{\partial f_m}{\partial x_1}&&\cdots&&\frac{\partial f_m}{\partial x_n}\end{matrix}\right),\\\mathbf{x_0}=x_i \mathbf{\hat{a}}_i,\mathbf{f}\left(\mathbf{x}\right)=f_i\left(\mathbf{x}\right)\mathbf{\hat{b}}_i$$

On the other hand, the gradient of $\mathbf{f}$ donoted by $\nabla\mathbf{f}$ is a linear transformation from $\mathbb{R}^m$ back to $\mathbb{R}^n$, defined, with respect to the same standard basis sets, so that the corresponding matrix of it is the $n \times m$ matrix

$$\left(\nabla\mathbf{f}\right)=\left(\begin{matrix}\frac{\partial f_1}{\partial x_1}&&\cdots&&\frac{\partial f_m}{\partial x_1}\\\vdots&&\ddots&&\vdots\\\frac{\partial f_1}{\partial x_n}&&\cdots&&\frac{\partial f_m}{\partial x_n}\end{matrix}\right)$$

Note, at least with respect to standard basis sets, that the gradient is the transpose of the total derivative.

The variation of function $\mathbf{f}$ at $\mathbf{x_0}$ in the direction of unit vector $\mathbf{u} \in \mathbb{R}^n$, i.e. the directional derivative of $\mathbf{f}$, denoted by $\mathrm{D}_\mathbf{u}\mathbf{f}\left(\mathbf{x_0}\right)$ is a vector in $\mathbb{R}^m$ given by applying the total derivative on $\mathbf{u}$,


In the special case of $m=1$, i.e. scalar valued function, the rhs of the equation above is a product of a $1\times n$ matrix and an $n\times1$ matrix, leading to a scalar. It happens that, in this particular case, the matrix product equals also to the dot product of the gradient of the function and the unit vector. That's why when talking about scalar functions the textbooks always link the gradient with the directional derivative by the dot product as rule of calculation. However, we cannot generalize directly the dot-product rule to vector valued functions.

About your appendix, if we use the tensor product of the base vectors of two vector spaces as the basis to express a linear transform between these two vector space, we must be careful about the dimensions. In fact in tensor analysis we already have a rigorous and general definition. But here suppose we redefined something only for the specific purpose describe by this question.

Let $\mathcal{V}_n$ denote an $n$-dimensional vector space on $\mathbb{R}$. A linear transformation $\mathbf{A}:\mathcal{V}_n\rightarrow\mathcal{W}_m$ has its $m\times n$ matrix representation $A_{ji}$ under basis sets $\left\{\mathbf{\hat{e}}_i\right\}\in\mathcal{V}_n$ and $\left\{\mathbf{\hat{f}}_i\right\}\in\mathcal{W}_m$ can be obtained by acting on the former by $\mathbf{A}$ to give $n$ vectors $\mathbf{u}_i=A_{ji}\mathbf{\hat{f}}_j $. When acting on a vector $\mathbf{c}\in\mathcal{V}_n$ we get $\mathbf{d}\in\mathcal{W}_m$, $\mathbf{d}=\mathbf{Ac}$. Under the basis set we have the matrix calculation of this transformation


or $d_j=A_{ji}c_i$.

On the other hand, if under a certain definition of tensor product of two vectors, the tensor product of $\mathbf{v}\in\mathcal{V}_n$ and $\mathbf{w}\in\mathcal{W}_m$ is expressed with respect to the same basis sets as $\mathbf{v}\otimes\mathbf{w}=v_i w_j \mathbf{\hat{e}}_i\otimes\mathbf{\hat{f}}_j$ the resulted tensor corresponds to the $n\times m$ matrix representation $v_iw_j$. To construct a tensor that can act on vectors of $\mathcal{V}_n$ by $\mathbf{v}$ and $\mathbf{w}$ we have to use $\mathbf{w}\otimes\mathbf{v}$.

Therefore, to express the linear transformation $\mathbf{A}$ by the two basis sets, it should be in the form $\mathbf{A}=A^\prime_{ij}\mathbf{\hat{f}}_i\otimes\mathbf{\hat{e}}_j$. To see the relation between the two $m\times n$ matrices $A_{ji}$ and $A^\prime_{ij}$ we applied again on vector $\mathbf{c}$, this time using the expression with $A^\prime_{ij}$, and requiring that the results to be $\mathbf{d}$. We get in this case $\mathbf{d}=A^\prime_{ij}c_k\left(\mathbf{\hat{f}}_i\otimes\mathbf{\hat{e}}_j\right)\mathbf{\hat{e}}_k$.

To proceed we need an addition postulation in the present discussion, that it is a rule that


Then, $\mathbf{d}=A^\prime_{ij}c_k\left(\mathbf{\hat{f}}_i\otimes\mathbf{\hat{e}}_j\right)\mathbf{\hat{e}}_k=A^\prime_{ij}c_k\mathbf{\hat{f}}_i\left(\mathbf{\hat{e}}_j\cdot\mathbf{\hat{e}}_k\right)$. We again have to end here unless in the special case that $\left\{\mathbf{\hat{e}}_i\right\}$ is orthonormal basis set. In this case $\mathbf{d}=A^\prime_{ij}c_j\mathbf{\hat{f}}_i$ or $d_i=A^\prime_{ij}c_j$. By a comparison and care on the subscripts we know that $A^\prime_{ij}=A_{ij},i=1,\cdots,n,j=1,\cdots,m$.

We can now conclude that the linear transformation $\mathbf{A}:\mathcal{V}_n\rightarrow\mathcal{W}_m$ can be expressed with respect of orthonormal basis sets $\left\{\mathbf{\hat{e}}_i\right\}\in\mathcal{V}_n$ and $\left\{\mathbf{\hat{f}}_i\right\}\in\mathcal{W}_m$ as $\mathbf{A}=A_{ij}\mathbf{\hat{f}}_i\otimes\mathbf{\hat{e}}_j$ under the rule $\left(\mathbf{w}\otimes\mathbf{v}\right)\mathbf{c}=\mathbf{w}\left(\mathbf{v}\cdot\mathbf{c}\right)$.

So the (total) derivative of function $\mathbf{f}$, $\mathrm{d}_\mathbf{x_0}\mathbf{f}$, i.e. the "correct" linear transformation we used to act on a unit vector to get a directional derivative, should be expressed as $\mathrm{d}_\mathbf{x_0}\mathbf{f}=\frac{\partial f_i}{\partial x_j}\mathbf{\hat{b}}_i\otimes\mathbf{\hat{a}}_j$ (since standard basis are orthonormal).


To generalize, let's first go back a little and talk about the directional derivative of a scalar-valued function $f(\vec{x})$ of a vector variable $\vec{x}$ in a general and invariant language. If $\vec{d}$ is a direction vector (unit length), then the directional derivative of $f$ at $\vec{x} = \vec{x}_{0}$ in the direction $\vec{d}$ can be defined as follows:

It is the image of the linear transformation ${df \over d\vec{x}}( \vec{x}_{0})$ acting on the vector $\vec{d}$.

Thus, the generalization consists in replacing the scalar funtion $f$ by a vector-valued one, $\vec{f}$, and writing down the invariant definition of the derivative $$ {d\vec{f} \over d\vec{x}}( \vec{x}_{0}). $$ This derivative is, by definition, a certain linear transformation from (the tangent space at $\vec{x}_{0}$ of the domain of $\vec{f}$) to (the tangent space at $\vec{f}(\vec{x}_{0})$ of the range of $\vec{f}$).

The specific defining properties of this linear transformation can (and should be at first) stated without resorting to bases or tensor representations, and are described on page 66 of this book: https://books.google.com/books?id=JUoyqlW7PZgC&printsec=frontcover&dq=arnold+ordinary+differential+equations&hl=en&sa=X&ved=0ahUKEwjGv_y44OfPAhXDSSYKHXvZCC4Q6AEIHjAA#v=onepage&q=The%20action%20of%20diffeomorphisms&f=false

  • $\begingroup$ Could you give an assessment for the Appendix section? $\endgroup$ Oct 20, 2016 at 15:55
  • $\begingroup$ Not sure what section you are referring to, and what you mean by assessment. $\endgroup$
    – user8960
    Oct 20, 2016 at 17:52
  • $\begingroup$ The OP (here: the original post) has a section titled Appendix. I was wondering whether the last formula makes sense (A). Also, what specifically is wrong with multiplying the gradient of a vector field with a unit vector? (B) $\endgroup$ Oct 20, 2016 at 18:52
  • $\begingroup$ (A): see my response to (B).:) (B) Not necessarily anything wrong. The gradient of a vector field is generally a linear transformation. So, we need to be specific about what we mean by "multiplying a vector by a linear transformation". This will determine whether the last formula in your Appendix makes sense. And this presence of sense is most conveniently examined using invariant definitions (i.e., those independent of a specific choice of a coordinate system). $\endgroup$
    – user8960
    Oct 20, 2016 at 18:57
  • 1
    $\begingroup$ Let us continue this discussion in chat. $\endgroup$
    – user8960
    Oct 21, 2016 at 18:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .