In the following picture, the author of the Field and Wave Electromagnetics shows the geometrical meaning of the direction of the gradient. That is, only by following the direction of the normal vector to the curve at that pointer could the rate of change be the maximum.

alt text

But what about the geometrical interpretation of the magnitude of gradient generally or maybe is there a geometrical interpreation of the magnitude of graident?



5 Answers 5


Short version of answer:

The gradient defines a direction; the magnitude of the gradient is the slope of your surface in that direction.

This direction just so happens to be the one in which you have to go to get the maximum slope.

Long version:

Let's say you take the gradient of an N surface in N+1 space. For instance, the gradient of a 2D surface in 3D space. The gradient will point in the direction that you have to go in to get the biggest increase in "height" (that +1 dimension). So, in other words, if you go in the direction in which the gradient points, you'll see the largest increase.

The magnitude of the gradient is the rate at which that increase happens. Literally, it is the slope of the surface at that point along the axis defined by the gradient's direction. Consequently, the magnitude of the gradient of some point on a surface is the steepest slope you can find on that surface!

Proof in 3D:

Gradients are actually defined to behave as described above, but, perhaps you are like me, and want a little bit of mathematical proof of this.

Let's start with the tangent plane to your surface at some point. If the above is true, then the magnitude of your gradient should equal the slope of the plane along the direction defined by your gradient! To start, let's quickly define the slope of a plane in a certain direction:

If we want to know the slope of a plane in a certain direction, we simply find the slope of said plane between the point (0, 0), and the point represented by our arbitrary direction. To simplify this procedure, we can shift our plane so that it passes through the point (0, 0, 0), since shifting a plane does not change its slope.

To recap, the slope of a plane in some 2D direction D is the same as the slope of a similar plane (which passes through (0,0,0))) between the points (0,0) and D.

The slope of a plane between 2D points (0,0) and D is given by:

$$ \frac {\frac{\partial z}{\partial x}D_x + \frac{\partial z}{\partial y}D_y}{||D||} $$


$$ \frac {\frac{\partial z}{\partial x}D_x + \frac{\partial z}{\partial y}D_y}{\sqrt{D_x^2 + D_y^2}} $$

Since this slope-in-a-direction is defined only in terms of the partial derivatives of our plane, and since the partial derivatives of any tangent plane to a surface are the same as those of the surface at the tangent point, we may make the claim that:

The slope of a surface at some point (x, y) in the direction D is given by the expression: $$ \frac {\frac{\partial f(x,y)}{\partial x}D_x + \frac{\partial f(x,y)}{\partial y}D_y}{\sqrt{D_x^2 + D_y^2}} $$

Meanwhile, the gradient of our surface at this point is given by the expression:

$$ {\frac{\partial f(x,y)}{\partial x} \hat{x} + \frac{\partial f(x,y)}{\partial y} \hat{y}} $$

which means that the slope of our surface at some point (x, y) in the direction of our gradient at that point is (as defined by our slope-in-direction argument earlier) given by:

$$ \frac {\frac{\partial f(x,y)}{\partial x} \cdot \frac{\partial f(x,y)}{\partial x} + \frac{\partial f(x,y)}{\partial y} \cdot \frac{\partial f(x,y)}{\partial y}}{\sqrt{\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2}} $$

This, of course, simplifies to:

$$ \frac {\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2}{\sqrt{\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2}} $$

Now for the fun part! Let's define some temporary variable J as:

$$ J = \left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2 $$

Then our "simplified" expression for the slope of our curve at any point (x,y) along its gradient at (x,y) becomes:

$$ \frac {J}{\sqrt{J}} $$

Which is the same thing as simply

$$ \sqrt{J} $$

Which is the same thing as

$$ \sqrt{\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2} $$

Which, but of course, by pythagorean theorem, is the same expression as the magnitude of our gradient at the point (x,y)!

And so, we have just proven that, yes, the magnitude of the gradient of a surface at some point is the same as the slope of that surface along said gradient.

I hope this helped!

Side note: Hans Lundmark's answer touches on this a bit; those curves made from the intersections of your volume in 4D space with planes at evenly spaced positions on the 4th axis (equivalent to the contour lines on a surface in 3D space) will, indeed, be closer together when the slope of your volume is steeper; this is because, as one might expect, we cross through more height-surfaces (and therefore more "vertical" distance) in an area which is more steep than in an area which is less steep. Hopefully the above slope explanation makes why that is (specifically, how it relates to the magnitude of the gradient) a bit clearer.

  • $\begingroup$ "Literally, it is the slope of the surface at that point along the axis defined by the gradient's direction." — what is slope here? A derivative at point is equal to tan of angle of tangent line. Does the same apply to gradient? Is its magnitude equal to tan of angle? $\endgroup$
    – koddo
    Commented Oct 7, 2018 at 15:57
  • $\begingroup$ I mean, directional derivative along gradient is a tan of angle, isn't it? $\endgroup$
    – koddo
    Commented Oct 7, 2018 at 16:23

If you look at the set of points satisfying f(x) = c, the gradient of f is normal to the surface and points in the direction of greatest increase of f. The magnitude of the gradient is proportional to the rate of increase.

  • 2
    $\begingroup$ @Jichao: this is more or less by definition. $\endgroup$ Commented Dec 2, 2010 at 17:14
  • $\begingroup$ I think you mean the gradient of f is normal to the surface f(x) = c, yes? $\endgroup$
    – user856
    Commented Dec 2, 2010 at 17:43
  • $\begingroup$ Yes it should read normal to the surface. $\endgroup$
    – Mykie
    Commented Dec 2, 2010 at 17:45
  • $\begingroup$ Thanks, I corrected my answer. It's easy to see the gradient points in the normal direction: there's no change in f along the surface, so the direction of greatest change points away from the surface. $\endgroup$ Commented Dec 2, 2010 at 20:23

Here's a geometric way of thinking that might be helpful: Consider a family of level surfaces $f(x,y,z)=C$ for some evenly spaced values of $C$ (where the spacing should be fairly small). These level surfaces will lie closely stacked in space near points where $|\nabla f|$ is large, and farther apart near points where $|\nabla f|$ is small.

(The two-dimensional counterpart is curves of constant elevation on a map; they are densely packed where the slope of the terrain is steep.)

  • 1
    $\begingroup$ I know this response is just about 4 years old, but have to say using the concept of stacking and spacing is an elegant way to interpret gradients and level surfaces. Easy as interpreting gradients to level curves by how "steep" the slope would be. $\endgroup$
    – user64878
    Commented Sep 29, 2014 at 6:45

If you are at a point ${\bf p}$ in the domain of $f$ then the rate of change of $f$ when starting from ${\bf p}$ to nearby points depends on the direction you take. The gradient $\nabla f({\bf p})$ is a vector attached at the point ${\bf p}$; it points into the direction of maximal positive rate of change of $f$. The actual rate of change in this direction is $|\nabla f({\bf p})|$, and in the opposite direction it is $-|\nabla f({\bf p})|$. For an arbitrary unit vector ${\bf e}$ attached at ${\bf p}$ the $directional\ derivative$ of $f$ in direction ${\bf e}$ is given by the scalar product $\nabla f({\bf p})\bullet {\bf e}$.

  • $\begingroup$ Very clear and intuitive. Thanks a lot. $\endgroup$
    – ZeroCool
    Commented Dec 24, 2015 at 13:50

If you're standing on a point on a hill, the gradient gives the direction of the steepest ascent from that location immediately outward. The magnitude of the gradient gives how steep that ascent is, by elementary slope.

If you work out the algebra, the rate of change in a function $f$ in the direction of the unit vector $\eta$ is $\nabla f \cdot \eta$ (This is simply the chain rule.). This quantity reaches its maximum when $\eta$ shares the same direction as $\nabla f$ (to maximize the cosine of the angle that's part of the dot product formula). Thus the magnitude of $\nabla f$ gives the rate of change in that direction, as well.

  • $\begingroup$ The analogy with the hill might be a bit confusing here, since it refers to the gradient of a two-variable function $f(x,y)$, the hill being its graph $z=f(x,y)$. The picture in the question, on the other hand, seems to illustrate the gradient of a three-variable function $f(x,y,z)$. $\endgroup$ Commented Dec 2, 2010 at 19:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .