Why the $\nabla f$ in the direction orthogonal to $f$?

I can't understand it intuitively.

Is it because of the definition of gradient?

  • 1
    $\begingroup$ What is $f(x)$? To discuss if $a\perp b$, we better have $a,b$ in the same vector space (with inner product). $\endgroup$ – Hagen von Eitzen May 25 '13 at 8:15
  • 2
    $\begingroup$ To make sense out of your question: Let $U\subset\mathbb R^n$ be an open set and $f\colon U\to\mathbb R$ continuously differentiable. Then if $\nabla f(x_0)\ne 0$, the contour $f^{-1}(f(x_0))$ is (near $x_0$) a 1-codimensional submanifold of $\mathbb R^n$ and approximated by the hyperplane through $x_0$ orthogonal to $\nabla f(x_0)$. To simplify things (hello intuition), consider the case that $f$ is linear where the contour is the hyperplane. $\endgroup$ – Hagen von Eitzen May 25 '13 at 8:24
  • 1
    $\begingroup$ $\nabla{f}(\vec{x})$ is not orthogonal to $f(\vec{x})$. I think what you mean to say is $\nabla{\vec{g}(z,\vec{x})}$ is orthogonal to $g(z,\vec{x})$, where $z=f(\vec{x})$ and $g(z,\vec{x})=z-f(\vec{x})$. $\endgroup$ – Ataraxia May 25 '13 at 9:11
  • 2
    $\begingroup$ I think you didn't quite ask what you meant to ask. I suspect that your real questions is this: take the surface $f(x,y,z) = 0$, where $f$ is a real-valued function with sufficiently nice properties. Then, as you said, at a given point $P_0 = (x_0, y_0, z_0)$, the vector $\nabla f(x_0, y_0, z_0)$ is perpendicular to the surface. You would like to have an intutitive and elementary explanation of why this is the case. Is that right? $\endgroup$ – bubba May 25 '13 at 10:37
  • 1
    $\begingroup$ First tell us what is $x$ and $f$, in the sense of scalar, vector or even tensor? My guess would be $x\in \mathbb{R}^n$ and $f:\mathbb{R}^n\rightarrow \mathbb{R}$. Check out wiki. To get a good intuition I would suggest you just take $x\in\mathbb{R}^2$. $\endgroup$ – newbie May 25 '13 at 12:17

The correct statement is that the gradient vector $\nabla f(x_0,y_0)=\left(\frac{\partial}{\partial x} f(x_0,y_0),\frac{\partial}{\partial y} f(x_0,y_0) \right)$ is orthogonal to any level curve $\mathbf{\alpha(t)=(x(t),y(t))}$ of the function $\mathbf{ f} $ in point $\alpha(t_0)=(x_0,y_0)$.

That is, the gradient vector $\nabla f(x_0,y_0)=\left(\frac{\partial}{\partial x} f(x_0,y_0),\frac{\partial}{\partial y} f(x_0,y_0) \right)$ of $f$ in point $(x_0,y_0)$ is orthogonal to the tangent vector $\alpha'(t_0)=\big(x'(t_0),y'(t_0)\big)$.

enter image description here

Let us explain this in a more detailed way. This in fact is the statement of a theorem of calculus in several variables.

Theorem. Let $f:\Omega\subset\mathbb{R}^2\to \mathbb{R}$ be a differentiable function in a open $\Omega\subset \mathbb{R}^2$ and suppose that all partial derivatives of $f$ are continuous. Consider a curve $\alpha:I\to \mathbb{R}^2$ differentiable in interval $I=(a,b)$ and living in
$$ f^{-1}(c)=\{ (x,y)\in\Omega : f(x,y)=c\}. $$ A curve with this property is called a level curve in $c$. Suppose that the curve $\alpha(t)=(x(t),y(t))$ have coordinates $x(t)$ and $y(t)$ whose derivatives $x'(t)$ and $y'(t)$ are continuous. For all $t_0\in I$ and $\alpha(t_0)=(x_0,y_0)$, $$ \langle \nabla f(x_0,y_0), \alpha'(t_0)\rangle = \frac{\partial}{\partial x} f(x_0,y_0)\cdot x'(t_0)+\frac{\partial}{\partial y} f(x_0,y_0)\cdot y'(t_0)=0 $$

Proof. The function $(a,b)\ni t\mapsto f\circ \alpha(t)= f\big(x(t),y(t)\big)\in\mathbb{R}$ is constant equal to $c$, that is, $f\circ \alpha(t)= f\big(x(t),y(t)\big)=c$. Looking $f\circ \alpha(t)$ as a function in the variable $t$ we know that $$ \frac{d}{dt}(f\circ \alpha)(t)= \frac{d}{dt} f\big(x(t),y(t)\big)=0. $$ On the other hand, by the chain rule and gradient property we have that $$ \left.\frac{d}{dt}(f\circ \alpha)(t)\right|_{t=t_0} = \frac{\partial}{\partial \vec{v}}f(x,y) = \langle \nabla f(x_0,y_0), \vec{v}\rangle = \langle \nabla f(\alpha(t_0)), \alpha'(t_0)\rangle $$ for $\alpha'(t_0)=(x'(t_0),y'(t_0))=(v_1,v_2)=\vec{v}$. Then $$ \langle \nabla f(x_0,y_0), \alpha'(t_0)\rangle = \frac{\partial}{\partial x} f(x_0,y_0)\cdot x'(t_0)+\frac{\partial}{\partial y} f(x_0,y_0)\cdot y'(t_0)=0 $$

  • 1
    $\begingroup$ Hi @Elias, how did you generate this image? Very good indeed, +1 for the complete explanation and for the image. $\endgroup$ – user1620696 May 25 '13 at 14:13
  • $\begingroup$ @user1620696 I generated an image but because of some bug it does not appear in my answer. The image is visible to you? $\endgroup$ – MathOverview May 25 '13 at 14:18
  • $\begingroup$ No, it really appears, but which software you've used to generate it? $\endgroup$ – user1620696 May 25 '13 at 14:19
  • 2
    $\begingroup$ @user1620696 I use Inkscape on Windows 7 platform. See the question in tex.stackexchange.com/questions/61274/… $\endgroup$ – MathOverview May 25 '13 at 14:21

The vector field $\operatorname{grad}(f)$ realizes the total derivative of $f$, in the sense that the inner product of $\operatorname{grad}(f)$ and a vector $v$ gives $df(v)$. But if $v$ is parallel to a level curve, then $df(v) = 0$, so it is perpendicular to the gradient.

Let $f:M\to \mathbb{R}$ be a smooth function with $n_0$ a regular value. Equip $M$ with a Riemannian metric. Suppose $f(m) = n_0$ and $v\in T_mM$ is parallel to the level set $f^{-1}(n_0)$. Then the trick is:

$$\langle \operatorname{grad}(f),v\rangle = df(v) = 0$$

because $v$ is annihilated by $df$.

So in a very real sense, this is a defining property of the gradient. We've constructed the gradient so that it is perpendicular to level sets.


Let's take an example to illustrate. Let $f(\boldsymbol{x}) = f(x,y)$. You may think $z = f(x,y)$ and you can plot the graph of the function in $\mathbb{R}^3$.

Now take a contour of $f(x,y)$, i.e. $f(x,y) = c$, total derivative shows $$\mathrm{d}f = \frac{\partial f}{\partial x}\mathrm{d}x+ \frac{\partial f}{\partial y}\mathrm{d}y = 0,$$ you can think $\mathrm{d}x, \mathrm{d}y$ as infinitesimal increments of $x,y$. With notice the above equation can be written as $$\nabla f\cdot\mathrm{d}\boldsymbol{x} = \begin{bmatrix} \frac{\partial f}{\partial x}\\ \frac{\partial f}{\partial y} \end{bmatrix} \cdot \begin{bmatrix} \mathrm{d} x\\ \mathrm{d} y \end{bmatrix} =0,$$ we have shown that $\nabla f(x,y)$ is orthogonal to $f(x,y) = c $ for all $(x,y)$ and $c$.

Let $\boldsymbol{v}$ be a unit vector, $\nabla f\cdot \boldsymbol{v}$ shows how $f$ would change in that direction. Notice $|\nabla f\cdot \boldsymbol{v}|\leq |\nabla f||\boldsymbol{v}| = |\nabla f| $ and equality only holds for $\boldsymbol{u} = \nabla f/|\nabla f|$. (You can argue what happen if $|\nabla f| = 0$, i.e. a plane $z = const.$)

So $\nabla f$ offers the information on how much and in which direction $f$ changes the most.

  • $\begingroup$ Is it true only for z=f(x,y) not for f(x,y) ? I think Zettasuro and bubba exactly pointed out what my question really was. $\endgroup$ – syko May 25 '13 at 12:45
  • $\begingroup$ @seongyun I took 2D case because you can actually plot it, with the value of $f(x,y)$, which I named as $z$. If going to higher dimensions, e.g. $f(x,y,z)$, the plot in $\mathrm{R}^3$, one can only plot the contour $f(x,y,z) = c$, and the gradient will gives you a vector orthogonal to the tangential plane of the surface of $f(x,y,z) = c$ at point$(x,y,z)$, with same argument. $\endgroup$ – newbie May 25 '13 at 12:47
  • $\begingroup$ @seongyun In short, $z$ is $f(x,y)$ and $f(x,y)$ is $z$. I thought it would have brought clarity. $\endgroup$ – newbie May 25 '13 at 12:50
  • 1
    $\begingroup$ @seongyun Aha. Intuition..$f(x,y) = c$ is where $f(x,y)$ does not change at all. The direction orthogonal to that, intuitively should be where $f(x,y)$ changes fastest. Does that make sense to you? $\endgroup$ – newbie May 25 '13 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.