I understand that the physical meaning of the gradient is a vector in the direction of maximum increase with the magnitude equivalent to the maximum rate of change. However, I'm simply not seeing it. Is there a good way of visualizing this?

Here is my thinking, which I am visualizing to be a counter example to the interpretation of the gradient. I'm not sure on how to graph this, so I'll explain in words. Assume a function T = x + y, where it is constant in the Z dimension (which allows me to put the T function on the z-axis). At the origin, the function increases equally and the most in both the x direction and the y direction, but along the line of y = x, the function decreases the most.

The gradient of this function at the origin, is the vector addition of the increase in the x direction and increase in the y direction. How then is the vector pointing in $T = <x, y>$ which is along the line of y = x, equivalent to the direction of maximum increase?

  • $\begingroup$ $T(x,y,z) = x+y$ would only be only be plottable in $4$d and wouldn't be constant in $z$. However, if you mean $T(x,y) = x + y$, then that is plottable in $3$d where $z = T(x,y)$. But again, it's not constant in $z$. $\endgroup$ – user137731 Feb 21 '17 at 4:59
  • $\begingroup$ think of a map that has the height contours - the gradient is the direction of the steepest change, up or down. As example, an electrical field has a direction in which the voltage change is highest. $\endgroup$ – Moti Feb 21 '17 at 5:00
  • $\begingroup$ The gradient of this function points "uphill" and has magnitude equal to the slope of the line in that direction. $\endgroup$ – user137731 Feb 21 '17 at 5:02
  • $\begingroup$ @Bye_World T(x,y,z) where z is constant is the same thing as T(x,y), right? $\endgroup$ – Goldname Feb 21 '17 at 5:10
  • $\begingroup$ @Bye_World That link you gives isn't the example that I am visualizing. I'm wondering what's wrong with my example. $\endgroup$ – Goldname Feb 21 '17 at 5:11

Consider two isotimic surfaces $u=c_1$ and $u=c_2$. Now consider the line of shortest distance between these two surfaces. The length of this line is the magnitude and its direction vector from the lower surface to the higher surface gives the direction of the gradient vector.

  • $\begingroup$ I'm not sure I understand what you mean by a surface u = c1 and v = c2? How is that a surface? $\endgroup$ – Goldname Feb 21 '17 at 5:09
  • $\begingroup$ Are you aware of the cartesian co-ordinates? What is $x=0$ or $y=0$? $\endgroup$ – SchrodingersCat Feb 21 '17 at 5:11
  • $\begingroup$ Ok I see what you mean now. But the partial derivative of any one of those surfaces is a tangent to that surface, so I don't see how the gradient vector of those two gives the shortest distance between the surface. $\endgroup$ – Goldname Feb 21 '17 at 5:13
  • $\begingroup$ So you want a proof? Check this. mathinsight.org/directional_derivative_gradient_derivation $\endgroup$ – SchrodingersCat Feb 21 '17 at 5:16
  • $\begingroup$ Not a proof. just something to visualize this. and also to figure out what's wrong with my specific example $\endgroup$ – Goldname Feb 21 '17 at 6:12

Here's one way to look at it, if you accept directional derivatives (also has a nice illustration of the gradient).

Suppose we are interested in the direction of greatest increase of some $f(x,y)$. Then, by definition we should look at the rate of increase of $f$ in the direction $v$ via its directional derivative: $$ R_f(v) = \nabla f\cdot \hat{v} $$ which is the rate of change of $f$ in the direction $v$. Intuitively, this is because you are projecting the total change (i.e. the gradient) of $f$ onto the direction vector $v$. It might be obvious that $R_f(v)$ is maximal when $v=\nabla f$, but you can easily see this by letting $$ \hat{v} = (\cos(\theta),\sin(\theta)) = \frac{(v_1,v_2)}{||(v_1,v_2)||_2} = \frac{v}{||v||_2}$$ so that (in 2D) $R_f(v) = \cos(\theta)\partial_xf + \sin(\theta)\partial_yf$. Notice that $\theta$ is simply the angle with the $x$ axis in the plane, so it determines the direction fully. Let's find the maximal $\theta$. Take $\partial_\theta R_f(v) = 0$ and you'll find $\theta=\arctan(\partial_y f/\partial_x f)$, which means $v = (v_1,v_2) = (\partial_x f, \partial_y f)=\nabla f$, as we expected. I.e. $ \max_v R_f(v) = (\nabla f\cdot \nabla f)/||\nabla f||_2 = ||\nabla f||_2 $. So the direction of maximal $R_f$ is $\nabla f$.

Ok, well for your specific example, $T(x,y)=x+y$, so $\nabla T = (1,1)$. How can we see that walking along the gradient gives the greatest increase? Let's just take a unit step (from the origin) in a few different directions to see why. Note that $T(0,0)=0$

First, in the $x$ direction, we take a step $s=(1,0)$, so $T(s)=1$. So we took a step of size $1$, and $T$ increased by 1.

Let's try in the $y$ direction: $s=(0,1)$, thus $T(s)=1$, so same as last time.

What about $s=(1,1)/\sqrt{2}$, which is a unit step in the gradient direction? Well, $T(s)=\sqrt{2} > 1$. So even though we took a step of size $1$, our function grew about ${\sim}{1.4}$, rather than just by $1$, as it did in the other directions.

What about something in between? Say, $s=(0.921,0.395)$, so then $T(s)=1.316$. So, taking a step of size $1$ led to an increase of ${\sim}1.3$. This is between the other values, as we would kind of expect.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.