It is well known that:

The directional derivative $\nabla_{{v}}{f}$ of a smooth function $f:\mathbb{R}^{n}\rightarrow \mathbb{R}$ in the direction of a vector $v$ is defined by: $$\nabla_{{v}}{f}({x}) = \lim_{h \rightarrow 0}{\frac{f({x} + h{v}) - f({x})}{h}}$$. We also know that the gradient of such an $f$ is defined implicitly by the equality $$\lim_{h\to 0} \frac{|f(x+h)-f(x) -\nabla f(x)\cdot h|}{|h|}=0$$.

Can we define (or derive from the latter definition) the gradient as a limit of a difference quotient?


Assuming $f$ is nice, the gradient at $x$ is the unique vector $\nabla f(x)$ such that for any direction $v$, $$ \nabla f(x)\cdot v = \nabla_v f(x)$$ So by knowing the directional derivatives in $n$ linearly independent directions (e.g. in the direction of the coordinate axes) we can deduce $\nabla f(x)$.

So if you write $F(t)$ for the vector valued function $F(t)_i = f(x+t e_i )$ where $e_i$ are the standard basis vectors, then this allows us to write $$ \nabla f(x) = F'(0) = \lim_{h\to 0} \frac{F(h)-F(0)}h$$

  • $\begingroup$ An explanation for the down vote would be appreciated $\endgroup$ – Calvin Khor Mar 6 '18 at 8:28
  • $\begingroup$ In your last equation can we replace zero with a minimum point $x^{*}$ of $f$? $\endgroup$ – johnny09 Feb 21 '19 at 1:28
  • $\begingroup$ @johnny09 I have no idea what you mean because the 0 there is a real number, and is the argument of $F$, not $f$ $\endgroup$ – Calvin Khor Feb 21 '19 at 8:48
  • $\begingroup$ The first order optimality condition states that $\nabla f(x^{*})=0$ must be satisfied for $x^{*}$ to be a minimum point. So, I was wondering if one could use your last equation with $x^{*}$. $\endgroup$ – johnny09 Feb 22 '19 at 2:17
  • $\begingroup$ There are no restrictions on $x$ in what I wrote, if that helps? @johnny09 $\endgroup$ – Calvin Khor Feb 22 '19 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.