# How can a gradient be thought of as a function?

If a function $$f(x,y)$$ would output a value in a third dimension, $$z=f(x,y)$$ for example. How can we treat the gradient of $$f$$ as a function in $$x$$ and $$y$$, when the output of the gradient is a vector in the two dimensions $$x$$ and $$y$$ which are the dimensions of the inputs. I guess my question is, is it normal for a function to map to the dimensions of its inputs?

• Problems in calculus often stem from physics and engineering, here we deal a lot with $\mathbb{R}^3$ and $\mathbb{R}^2$, so it makes sense that we visualise this by writing out a certain function in terms of $x,y,z$ we could define other basis vectors and write out some other functions for instance in terms of spherical polar coordinates, but the input is hard to visualise. We often transform to other coordinates to do integrations, for instance when we want to integrate over a sphere. The corresponding problem then transforms to some other graph in terms of these new coordinates. – user459879 Nov 9 '18 at 19:22
• As a function that does not depends on cartesian coordinates, consider the set {apple, smiley, (1,0), {{{1}}} } and some randomly chosen function that maps all of the natural numbers to each of these elements. We then have an infinite amount of elements mapped to 4 elements. This function could be surjective, but it is certainly not injective. I think my confusing in answering your question is also what your notion of "dimension" is, for me it means the amount of basis vectors in a vector space. – user459879 Nov 9 '18 at 19:58
• Functions do not need to be defined for vector spaces and certainly the cardinalities do not need to be the same. But we have a really nice set in real analysis, namely $\mathbb{R}$, which has a lot of nice properties. We often work with a cartesian product of this set with itself which is isomorphic to cartesian space. But I am just not sure what you mean with your question I guess. – user459879 Nov 9 '18 at 20:02

If a function $$f(x,y)$$ would output a value in a third dimension, $$z=f(x,y)$$ for example.

Generally that is not how a function of multiple variables works. If you happen to have a function that takes two real numbers as input and produces a single real number as its output, then you can plot the function in three dimensions, using two dimensions for the input and one for the output. To consider this as the definition of a function, however, or even to consider it as a "typical" function, is a mistake.

... is it normal for a function to map to the dimensions of its inputs?

Yes, it is "normal" in the sense that you will often encounter perfectly good functions with that property. There are also perfectly good functions that map to fewer dimensions than their input, and perfectly good functions that map to more dimensions than their input.

As an example of the "more dimensions" case, consider the position of a particle in space as a function of time: one input dimension, three output dimensions.

• I think historically people have thought of functions as things that can be plotted and even required to be smooth or continuous. Over time our definition of function has broadened significantly. – qwr Nov 10 '18 at 6:23
• "Over time" spans several hundred years in this case. I think you'd have to dig back pretty far to find someone who didn't recognize a parameterized curve as a function from $\mathbb R$ to $\mathbb R^2.$ And I'm not aware of any research-level mathematics that considered the plot of a function to be the function, or even to represent the dimensions in which the functions input and output exist by definition. – David K Nov 10 '18 at 6:51
• Sure, I didn't intend to formally define a function as such. It's just that usually functions geometrically introduce dimensions that are different from the input dimensions, either more or less dimensions but i mean physically they would have different meanings. However in the gradient function the output dimensions are the same cartesian dimensions that are inputs to the function we are differentiating. – Omar Hossam Ahmed Nov 10 '18 at 22:34

A function $$F$$ can be thought of as a machine that takes in inputs from one set, say $$X$$, and outputs elements in another set, say $$Y$$. A function outputs a single element for every input, and we write it $$F : X \rightarrow Y$$.

In your case, the gradient of $$f$$ is just a function $$\nabla f : \mathbb{R}^2 \rightarrow \mathbb{R}^2$$, where for each input $$(x,y)$$, it outputs the element $$\left( \frac{\partial f}{\partial x}(x,y), \frac{\partial f}{\partial y}(x,y) \right)$$.

• @OmarHossamAhmed If you feel that an answer has sufficiently responded to your question, you can click the check mark to mark it as accepted. – Sambo Nov 9 '18 at 23:13

$$(1)$$ The gradient is a vector-valued function, this maps pairs of numbers $$(x,y) \in \mathbb R^2$$ to some other pair of numbers $$(x',y')\in \mathbb R^2$$. These pairs of numbers we call vectors and they have a very geometric interpretation: they have a length and a direction. Specifically the gradient corresponds to the direction and magnitude of steepest ascent.

Also see:

https://en.wikipedia.org/wiki/Vector-valued_function

$$(2)$$ On the contrary, when you plot a function that maps from $$\mathbb{R^2} \rightarrow \mathbb{R}$$ like $$f(x,y)=x^2 +y^2$$, when you want to plot this you often define a third variable $$z=f(x,y)$$ and you let the value of this variable be equal to the function value. I have plotted $$x^2 +y^2$$ in this way below:

Without introducing another axis, we can also just give different function value ranges a different colour, so a large value could be very dark and a low value could be very light or vice versa. We recognise the same function:

The difference between $$(1)$$ and $$(2)$$ is the notion of direction. The function you describe is usually a "scalar field", we only have the notion of magnitude or "value" but not of direction. Gradients will give you a so-called "vector field" as physicists often call it, we usually visualise this using VectorPlots. Below you find such a method, I've plotted the vector field $$f(x,y)=(x^2+y,y^2+x )$$

which has gradient $$grad(f)(x,y)=(2x,2y)$$

The gradient operator is a higher order function: it maps functions to functions. In case of scalar fields on real vector spaces, $$\nabla :(\mathbb{R}^n\to\mathbb{R}) \to (\mathbb{R}^n\to\mathbb{R}^n).$$ Thus, if $$F:\mathbb{R}^2\to\mathbb{R}$$, then $$\nabla F : \mathbb{R}^2 \to \mathbb{R}^2$$, and if you evaluate that at some point, you get a single vector, like $$\nabla F(x,y) : \mathbb{R}^2$$. Specifically, $$\nabla F(x,y) = \begin{pmatrix}\frac{\partial F(x,y)}{\partial x} \\ \frac{\partial F(x,y)}{\partial y}\end{pmatrix}.$$ See also What does the symbol nabla indicate?

• I never thought of it as a higher order function. Thanks for the insight! – Omar Hossam Ahmed Nov 10 '18 at 22:29

Tensors play an important role here. Examples include the usual vectors in three dimensional space. Matrices.

A tensor in a general sense is a function that takes vectors or one-forms and returns a real number. Further, the tensor is a linear function of these inputs. As with any other function, you can take the derivative of a tensor.

For example, suppose you have a rotation matrix that rotates point in the xy plane about the z axis. The matrix is a tensor. It's input is a vector. Linear changes in the input result in linear changes in the output.

In the case of a rotation matrix, there are 2 inputs in effect, how much to rotate by, and the initial position. So you can think of the final position as a vector function of x,y, and $$\theta$$, and you can consider derivatives with respect to those.

Consider $$f(x,y,z)=ax^2+by^2+cz^2=1$$, with $$a=b=c=1$$. What does $$\nabla f$$ look like? It's essentially double the position vector pointing from the origin to the point. Any infinitesimal displacement tangent to the surface will be perpendicular to the gradient.

$$df=\nabla f \cdot d\vec{s}$$

If $$|df|>0$$, then you are necessarily leaving the surface.

As you change $$a,b,c$$, you get a different surface and a different gradient. Always perpendicular to the surface. This is another sense in which it makes sense to consider a derivative of a gradient.