Why is the gradient always perpendicular to level curves?

Question

Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be a function. A level set is a set of points:

$$L(c) = \{x \in \mathbb{R}^n | f(x) = c\}$$

Two vectors $a, b \in \mathbb{R}^n$ are perpendicular, when their dot product is 0: $$a \perp b :\Leftrightarrow a \cdot b = \sum_{i=1}^n a_i b_i = 0$$

The gradient of $f$ is

$$\nabla f = \begin{pmatrix}\frac{\partial f}{\partial x_1}\\ \frac{\partial f}{\partial x_2}\\ \dots\\ \frac{\partial f}{\partial x_n}\\\end{pmatrix}$$

Question

• Why is $\nabla f(p)$ at any given point $p \in \mathbb{R}^n$ perpendicular to the level set $L(f(p))$?
  • What does it mean anyway to be perpendicular to the level set? Does it mean the tangent of the level set in this point is perpendicular to the gradient in this point?
    • How do I get the tangent?
  • Are there any important implications of this?

Context

I found the question "why is the level curve perpendicular to the gradient" in an exam protocol for probabilistic planning.

Answer 1

First of all, when dealing with more than two variables level set is a better denomination than level curve (or level surface in three dimensions.)

Now to your question. Let $x_0\in L(c)$ and let $\gamma\colon(-a,a)\to \mathbb{R}^n$ be a $C^1$ curve contained in $L(c)$ and such that $\gamma(0)=x_0$. Then $$ f(\gamma(t))=c,\quad -a<t<a. $$ Differentiating with respect to $t$ and evaluating at $t=0$ we get $$ \nabla f(x_0)\cdot\gamma'(0)=0. $$ The set of all vectors $\gamma'(0)$ for all possible curves $\gamma$ forms the tangent hyperplane to $L(c)$ at $x_0$, and $\nabla f(x_0)$ is orthogonal to all of them, that is, the gradient is orthogonal to the tangent hyperplane of the level set.

Answer 2

For your first question:

We say a vector $v \in R^n$ is perpendicular to $S \subset R^n$ at $p \in S$ if for every curve $\gamma: (-a,a) \to R^n $ s.t $Img(\gamma) \subset S$ and $\gamma(0) = p$, it holds that $\gamma'(0) \cdot v = 0$

With this definition:

Suppose that $\bigtriangledown f(p) \neq 0$

Let $\gamma:(-a,a) \to L(f(p))$ with $\gamma(0) = p$.

Notice that $\forall x \in L(f(p))$ $f(x) = f(p)$ is constant.

Thus $\forall t \in (-a,a)$ $f(\gamma(t)) = f(p)$, and we can differentiate to get $ 0 = \frac{d}{dt}f(\gamma(t)) = \bigtriangledown f(\gamma(t))\gamma'(t)$.

Setting $t = 0 $ yields $ 0 = \bigtriangledown f(\gamma(0))\gamma'(0) = \bigtriangledown f(p)\gamma'(0) $ which is what we wanted.