Vector normal to the gradient I know that the gradient points in the direction of the maximum rate of increase, with the maximum rate given by its magnitude.  Similarly, the maximum rate of decrease is given by the negative of its magnitude.  Does a vector normal to the gradient point in the direction of zero increase?  I.e., if I were standing on a level curve and traveled in a direction normal to the gradient at that point, will I land on a level curve with the same value as when I began?
 A: Yes. We can define the directional derivative of $f$ at the point $\mathbf r$ along a unit vector $\mathbf n$ by $$\mathbf n\cdot\nabla f=\lim_{h\to0}\frac{1}{h}\left[f(\mathbf r+h\mathbf n)-f(\mathbf r)\right].$$
If $\mathbf n$ is perpendicular to $\nabla f$, then the directional derivative is $0$, so $f$ doesn't change when you move in the direction of $\mathbf n$.
A: Generic Case
If a function $f$ is nice enough that it is differentiable at a point $\mathbf{x}$, the directional derivative in the direction of $\mathbf{v}$ (at least if $\mathbf{v}$ is a unit vector), is given by the dot product of the gradient and $\mathbf{v}$: $\nabla_{\mathbf{v}}f(\mathbf{x})=\nabla f(\mathbf{x})\bullet \mathbf{v}$. If you are working in at least two dimensions, you can do as jlammy suggested and take $\mathbf{v}$ in a direction perpendicular to $\nabla f(\mathbf{x})$ to force $\nabla f(\mathbf{x})\bullet \mathbf{v}=0$, so that the directional derivative is zero and there is no instantaneous increase or decrease.
Exception
However, not every function is differentiable, and it is possible for every directional derivative to be nonzero (including the direction perpendicular to the gradient). See my answer to "Is there always a direction in which the directional derivative of a function is zero?" for an example.
Caveat

I.e., if I were standing on a level curve and traveled in a direction normal to the gradient at that point, will I land on a level curve with the same value as when I began?

However, just because the directional derivative is $0$ doesn't mean you can end up at the same value. For instance, the derivative of $f(x)=x^3$ is $0$ at $x=0$, but $f$ is increasing through the point $(0,0)$, and so walking on that graph can't land at height $0$ at more than one spot.
