Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
In the following picture, the author of the Field and Wave Electromagnetics shows the geometrical meaning of the direction of the gradient. That is, only by following the direction of the normal vector to the curve at that pointer could the rate of change be the maximum.
But what about the geometrical interpretation of the magnitude of gradient generally or maybe is there a geometrical interpreation of the magnitude of graident?
thanks.
The gradient defines a direction; the magnitude of the gradient is the slope of your surface in that direction.
This direction just so happens to be the one in which you have to go to get the maximum slope.
Let's say you take the gradient of an N surface in N+1 space. For instance, the gradient of a 2D surface in 3D space. The gradient will point in the direction that you have to go in to get the biggest increase in "height" (that +1 dimension). So, in other words, if you go in the direction in which the gradient points, you'll see the largest increase.
The magnitude of the gradient is the rate at which that increase happens. Literally, it is the slope of the surface at that point along the axis defined by the gradient's direction. Consequently, the magnitude of the gradient of some point on a surface is the steepest slope you can find on that surface!
Gradients are actually defined to behave as described above, but, perhaps you are like me, and want a little bit of mathematical proof of this.
Let's start with the tangent plane to your surface at some point. If the above is true, then the magnitude of your gradient should equal the slope of the plane along the direction defined by your gradient! To start, let's quickly define the slope of a plane in a certain direction:
If we want to know the slope of a plane in a certain direction, we simply find the slope of said plane between the point (0, 0), and the point represented by our arbitrary direction. To simplify this procedure, we can shift our plane so that it passes through the point (0, 0, 0), since shifting a plane does not change its slope.
To recap, the slope of a plane in some 2D direction D is the same as the slope of a similar plane (which passes through (0,0,0))) between the points (0,0) and D.
The slope of a plane between 2D points (0,0) and D is given by:
$$ \frac {\frac{\partial z}{\partial x}D_x + \frac{\partial z}{\partial y}D_y}{||D||} $$
Or:
$$ \frac {\frac{\partial z}{\partial x}D_x + \frac{\partial z}{\partial y}D_y}{\sqrt{D_x^2 + D_y^2}} $$
Since this slope-in-a-direction is defined only in terms of the partial derivatives of our plane, and since the partial derivatives of any tangent plane to a surface are the same as those of the surface at the tangent point, we may make the claim that:
The slope of a surface at some point (x, y) in the direction D is given by the expression: $$ \frac {\frac{\partial f(x,y)}{\partial x}D_x + \frac{\partial f(x,y)}{\partial y}D_y}{\sqrt{D_x^2 + D_y^2}} $$
Meanwhile, the gradient of our surface at this point is given by the expression:
$$ {\frac{\partial f(x,y)}{\partial x} \hat{x} + \frac{\partial f(x,y)}{\partial y} \hat{y}} $$
which means that the slope of our surface at some point (x, y) in the direction of our gradient at that point is (as defined by our slope-in-direction argument earlier) given by:
$$ \frac {\frac{\partial f(x,y)}{\partial x} \cdot \frac{\partial f(x,y)}{\partial x} + \frac{\partial f(x,y)}{\partial y} \cdot \frac{\partial f(x,y)}{\partial y}}{\sqrt{\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2}} $$
This, of course, simplifies to:
$$ \frac {\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2}{\sqrt{\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2}} $$
Now for the fun part! Let's define some temporary variable J as:
$$ J = \left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2 $$
Then our "simplified" expression for the slope of our curve at any point (x,y) along its gradient at (x,y) becomes:
$$ \frac {J}{\sqrt{J}} $$
Which is the same thing as simply
$$ \sqrt{J} $$
Which is the same thing as
$$ \sqrt{\left(\frac{\partial f(x,y)}{\partial x}\right)^2 + \left(\frac{\partial f(x,y)}{\partial y} \right)^2} $$
Which, but of course, by pythagorean theorem, is the same expression as the magnitude of our gradient at the point (x,y)!
And so, we have just proven that, yes, the magnitude of the gradient of a surface at some point is the same as the slope of that surface along said gradient.
I hope this helped!
Side note: Hans Lundmark's answer touches on this a bit; those curves made from the intersections of your volume in 4D space with planes at evenly spaced positions on the 4th axis (equivalent to the contour lines on a surface in 3D space) will, indeed, be closer together when the slope of your volume is steeper; this is because, as one might expect, we cross through more height-surfaces (and therefore more "vertical" distance) in an area which is more steep than in an area which is less steep. Hopefully the above slope explanation makes why that is (specifically, how it relates to the magnitude of the gradient) a bit clearer.
If you look at the set of points satisfying f(x) = c, the gradient of f is normal to the surface and points in the direction of greatest increase of f. The magnitude of the gradient is proportional to the rate of increase.
Here's a geometric way of thinking that might be helpful: Consider a family of level surfaces $f(x,y,z)=C$ for some evenly spaced values of $C$ (where the spacing should be fairly small). These level surfaces will lie closely stacked in space near points where $|\nabla f|$ is large, and farther apart near points where $|\nabla f|$ is small.
(The two-dimensional counterpart is curves of constant elevation on a map; they are densely packed where the slope of the terrain is steep.)
If you are at a point ${\bf p}$ in the domain of $f$ then the rate of change of $f$ when starting from ${\bf p}$ to nearby points depends on the direction you take. The gradient $\nabla f({\bf p})$ is a vector attached at the point ${\bf p}$; it points into the direction of maximal positive rate of change of $f$. The actual rate of change in this direction is $|\nabla f({\bf p})|$, and in the opposite direction it is $-|\nabla f({\bf p})|$. For an arbitrary unit vector ${\bf e}$ attached at ${\bf p}$ the $directional\ derivative$ of $f$ in direction ${\bf e}$ is given by the scalar product $\nabla f({\bf p})\bullet {\bf e}$.
If you're standing on a point on a hill, the gradient gives the direction of the steepest ascent from that location immediately outward. The magnitude of the gradient gives how steep that ascent is, by elementary slope.
If you work out the algebra, the rate of change in a function $f$ in the direction of the unit vector $\eta$ is $\nabla f \cdot \eta$ (This is simply the chain rule.). This quantity reaches its maximum when $\eta$ shares the same direction as $\nabla f$ (to maximize the cosine of the angle that's part of the dot product formula). Thus the magnitude of $\nabla f$ gives the rate of change in that direction, as well.
