# Why partial derivations of scalar-valued function put into vector give direction of maximum increase?

I'm having trouble understanding why we can obtain direction of maximum increase(gradient) from simply using the partial derivatives as components of the vector? What reasoning is behind this? How it was discovered / proven that standing on inclined plane, the relation between the directional derivative in angle x and the angle x is a sinusoid (i gather this must have been known to the man who started to use dot product to calculate directional derivative from gradient)? Thanks in advance

Edit: to simplify - Why partial derivations of scalar-valued function put into vector give direction of maximum increase?

• Think about why this works when $f(x,y)$ is a function of $x$ or $y$ only. – Jim Belk Jun 21 '13 at 16:27

The big idea here is this: assuming that $f$ is "nice enough", the directional derivative $D_{\vec{v}}f(\vec{x})$ in the direction $\vec{v}$ (which is a unit vector) can be expressed by $$D_{\vec{v}}f(\vec{x})=\vec{v}\cdot\nabla f(\vec{x}),$$ where $\nabla f=\langle\frac{\partial}{\partial x_1}f,\ldots,\frac{\partial}{\partial x_n}f\rangle$ is the gradient of $f$. Why? The rough reason is that if $f$ is differentiable, then $$f(\vec{x}+\Delta\vec{x})\approx f(\vec{x})+\nabla f(\vec{x})\cdot\Delta\vec{x}.$$ (This is the multivariate generalization of the linear approximation, and can be though of as following from using the linear approximation in each coordinate.) Then by definition, for $h$ small the directional derivative is $$D_{\vec{v}}f(\vec{x})\approx\frac{f(\vec{x}+h\vec{v})-f(\vec{x})}{h}\approx\frac{\nabla f(\vec{x})\cdot h\vec{v}}{h}=\nabla f(\vec{x})\cdot\vec{v}.$$ (Obviously, there's some formalization needed here; but, I hope this gives you the idea!)
Now, if you recall, a dot product $\vec{v}\cdot\vec{w}$ can be written as $$\vec{v}\cdot\vec{w}=\|\vec{v}\|\,\|\vec{w}\|\cos\theta,$$ where $\theta$ is the angle between $\vec{v}$ and $\vec{w}$. In our specific case, this says $$D_{\vec{v}}f(\vec{x})=\|\nabla f(\vec{x})\|\cos\theta,$$ where $\theta$ is the angle between $\vec{v}$ and $\nabla f$. The only part of this expression that depends on $\vec{v}$ is $\cos\theta$; so, we maximize the directional derivative by maximizing $\cos\theta$, and minimize the directional derivative by minimizing $\cos\theta$. These happen when $\theta=0$ and $\theta=\pi$, respectively - corresponding to $v$ being either in the same direction as $\nabla f$ or the opposite direction.
Let's assume that we have a function of two variables $f(x_1,x_2)$. The change in any direction can be expressed as $$df=f(x_1+h_1,x_2+h_2)-f(x_1,x_2)$$ By using Taylor expansion for $f(x_1+h_1,x_2+h_2)$ the change becomes $$df=f(x_1,x_2)+\frac{\partial f(x_1,x_2)}{\partial x_1}h_1+\frac{\partial f(x_1,x_2)}{\partial x_2}h_2-f(x_1,x_2)$$ The rate of change can be written as $$\frac{df}{\sqrt{h_1^2+h_2^2}}=\frac{\partial f(x_1,x_2)}{\partial x_1}\frac{h_1}{\sqrt{h_1^2+h_2^2}}+\frac{\partial f(x_1,x_2)}{\partial x_2}\frac{h_2}{\sqrt{h_1^2+h_2^2}}$$ Since we want to maximize the rate of change wrt $h_1$ and $h_2$ we set the first derivatives to zero $$\frac{\partial f(x_1,x_2)}{\partial x_1}\Bigg(\frac{1}{\sqrt{h_1^2+h_2^2}}-\frac{h_1^2}{(h_1^2+h_2^2)^{3/2}}\Bigg)-\frac{\partial f(x_1,x_2)}{\partial x_2}\frac{h_1h_2}{(h_1^2+h_2^2)^{3/2}}=0$$ $$-\frac{\partial f(x_1,x_2)}{\partial x_1}\frac{h_1h_2}{(h_1^2+h_2^2)^{3/2}}+\frac{\partial f(x_1,x_2)}{\partial x_2}\Bigg(\frac{1}{\sqrt{h_1^2+h_2^2}}-\frac{h_2^2}{(h_1^2+h_2^2)^{3/2}}\Bigg)=0$$ Multipliying both equations by $(h_1^2+h_2^2)^{3/2}$ $$\frac{\partial f(x_1,x_2)}{\partial x_1}h_2^2-\frac{\partial f(x_1,x_2)}{\partial x_2}h_1h_2=0$$ $$-\frac{\partial f(x_1,x_2)}{\partial x_1}h_1h_2+\frac{\partial f(x_1,x_2)}{\partial x_2}h_1^2=0$$ As can be seen the equations are satisfied if $$h_1=\frac{\partial f(x_1,x_2)}{\partial x_1}\text{ and } h_2=\frac{\partial f(x_1,x_2)}{\partial x_2}$$ which is simply the gradient.