1
$\begingroup$

edit 2(final answer in terms of python)

in terms of programming:

def function_1(a,b):
    return 2a + 2b

def function_2(x,y):
    return function_1(xy,y)

  # mathematically, we have function_1(a(x,y),b(y))

def partial_fn1_wrt_a(a, b):
    h = 0.000000000...1
    return (function_1(a + h, b) - function_1(a, b))/h

def total_fn1_wrt_y(x, y):
    h = 0.000000000...1
    return (function_2(x, y + h) - function_2(x, y))/h
    # or perhaps more intuitively as:
    return (function_1(a(x, y + h), b(y + h)) - function_1(a(x,y), b(y))/h

def partial_fn1_wrt_y(x,y):
    # does not "exist" since partial, by definition only varies the
    # params, so this would be in the job domain of the total.
    # however many ppl, including me, get confused because it 
    # "exists" because ppl find it with the total, expand everything
    # out with the chain rule - and often collapse everything back 
    # together into another partial for convenience. but its not like
    # "technically correct" - because its more of an amalgamation of    
    # the partial and the total. the total wearing a sort of mask - 
    # if you will.
    #
    # confusion is often also compounded by the fact irl, both 
    # function_1 and function_2 would have identically named params
    # x, y for example. so it becomes vague if we are varying the 
    # actual underlying variable named x/y or the parameter of the 
    # current function that also happens to be named x/y

    # (please feel free to add to this if you like)

edit after ORION pointed out obvious error in original examples

it was pointed out by orion that there is an error in my previous example, so to really illustrate my point here is another what about the case where it is not differentiating wrt to the original parameters by to another exogenous variable?

example 1) $f = f(x,y,z)$ where only x is dependent on t

  • total derivative:

$$\frac {df} {dt} = \frac {\partial f} {\partial x} \frac {dx} {dt} + \frac {\partial f} {\partial y} \frac {dy} {dt} + \frac {\partial f} {\partial z} \frac {dz} {dt}$$ $$\frac {df} {dt} = \frac {\partial f} {\partial x} \frac {dx} {dt} + \frac {\partial f} {\partial y} (0) + \frac {\partial f} {\partial z} (0)$$ $$\frac {df} {dt} = \frac {\partial f} {\partial x} \frac {dx} {dt}$$

  • partial derivative $$\frac {\partial f} {\partial t} = \frac {\partial f} {\partial x} \frac {\partial x} {\partial t}$$

example 2) $f = f(x,y,z)$ where all x,y,z are dependent on t

  • total derivative:

$$\frac {df} {dt} = \frac {\partial f} {\partial x} \frac {dx} {dt} + \frac {\partial f} {\partial y} \frac {dy} {dt} + \frac {\partial f} {\partial z} \frac {dz} {dt}$$

  • partial derivative $$\frac {\partial f} {\partial t} = \frac {\partial f} {\partial x} \frac {\partial x} {\partial t} + \frac {\partial f} {\partial y} \frac {\partial y} {\partial t} + \frac {\partial f} {\partial z} \frac {\partial z} {\partial t}$$

now, arent they still the same? I really dont know when to use d vs $\partial$. the usage seems very similar to me. Looking at the preceding equations, I feel as if they are interchangeable and express the same thing. why wont the above examples work? and is there any text which helps in understanding this stuff, written in English and not archaic math jargon?


I am curious what the functional difference between partial and total derivatives are. like what can totals do that partials cant?

I know partials hold all variables constant except for the given whereas the totals just assumes everything is a function of the given

but the latter just seems like a semantic, esoteric generalization - since if the other variables really aren't a function of the given, they will get reduced to 0 all the same - resulting in the same partial. And since in real life, 99% of the time we already know what is a function of what, it feels like the total derivative is just a "safe/generalized" way at expressing a concept with negligible benefits - used in situations that in the real world is often just handled by partials

(note that I am ignoring the obvious case where f(x) is only a function of one variable, where you use d instead of $\partial$)

example:

case 1) $f = f(x,y,z)$ where we know that x,y,z are INDEPENDENT of each other

  • the total derivative: $$\frac {df} {dx} = \frac {\partial f} {\partial x} \frac {dx} {dx} + \frac {\partial f} {\partial y} \frac {dy} {dx} + \frac {\partial f} {\partial z} \frac {dz} {dx}$$ $$\frac {df} {dx} = \frac {\partial f} {\partial x} (1) + \frac {\partial f} {\partial y} (0) + \frac {\partial f} {\partial z} (0)$$ $$\frac {df} {dx} = \frac {\partial f} {\partial x}$$

  • the partial: $$\frac {\partial f} {\partial x} = \frac {\partial f} {\partial x}$$

in this case, we see that the partial derivative = the total derivative with the only difference being d and $\partial$

case 2) $f = f(x,y,z)$ where we know that x,y,z are DEPENDENT on x

  • the total derivative: $$\frac {df} {dx} = \frac {\partial f} {\partial x} \frac {dx} {dx} + \frac {\partial f} {\partial y} \frac {dy} {dx} + \frac {\partial f} {\partial z} \frac {dz} {dx}$$ $$\frac {df} {dx} = \frac {\partial f} {\partial x} (1) + \frac {\partial f} {\partial y} \frac {dy} {dx} + \frac {\partial f} {\partial z} \frac {dz} {dx}$$ $$\frac {df} {dx} = \frac {\partial f} {\partial x} + \frac {\partial f} {\partial y} \frac {dy} {dx} + \frac {\partial f} {\partial z} \frac {dz} {dx}$$

  • the partial: $$\frac {\partial f} {\partial x} = \frac {\partial f} {\partial x} + \frac {\partial f} {\partial y} \frac {\partial y} {\partial x} + \frac {\partial f} {\partial z} \frac {\partial z} {\partial x}$$

in this case, AGAIN, we see that the partial derivative = the total derivative with the only difference being d and $\partial$


in both cases the results are the same, given that we know about our function (which arguably we will in 99% of cases irl) - in machine learning for instance, since it is us that ends up designing the functions in the first place...

so is the difference between d and $\partial$ just something that mathematicians and really mathy people like to flock over, with no real significance outside of semantics?

$\endgroup$
  • 1
    $\begingroup$ In the second case, it is pretty clear that the partial derivative is not equal to the total derivative. Your question pretty much answers itself... $\endgroup$ – 5xum Jan 25 '16 at 8:28
5
$\begingroup$

The difference is very significant and used in everyday physics. Consider a practical example: equation of motion for a fluid is basically a Newton's law for a small volume of fluid. That is,

$$\rho \frac{d{\bf v}}{dt}=-\nabla p$$ where $\nabla p= (\frac{\partial p}{\partial x},\frac{\partial p}{\partial y},\frac{\partial p}{\partial z})$. Here, $\bf v$ is the velocity vector, $\rho$ is the density and $p$ is pressure (this is just mass*acceleration = force). I neglected the viscosity because that's not the point here.

So what does the derivative on the left mean? It's the total derivative over time: the change of velocity of that particular piece of fluid over time. But... let's expand that:

$$\frac{d{\bf v}}{dt}=\frac{\partial {\bf v}}{\partial t} + \frac{\partial {\bf v}}{\partial x}\frac{d x}{d t}+ \frac{\partial {\bf v}}{\partial y}\frac{d y}{d t}+ \frac{\partial {\bf v}}{\partial z}\frac{d z}{d t}$$ The first term on the right is how much the velocity at that particular fixed point in space change (x,y,z being held fixed), and thus does not take into account the movement of fluid itself. The rest of the terms are advection: the velocity of the fluid particle changes not only because the velocity in its position changes, but also because the fluid moves to a place where velocity is different. You may recognize the velocity components on the right:

$$\frac{d{\bf v}}{dt}=\frac{\partial {\bf v}}{\partial t} + \frac{\partial {\bf v}}{\partial x}v_x+ \frac{\partial {\bf v}}{\partial y}v_y+ \frac{\partial {\bf v}}{\partial z}v_z=\frac{\partial {\bf v}}{\partial t}+({\bf v}\cdot \nabla){\bf v}$$

The second term is nonlinear (velocity squared!) and is the main reason fluid dynamics is so difficult! This nonlinearity leads to chaotic motion (turbulence) which makes analytical predictions mostly impossible, and numerical simulations unreliable for longer times (weather forecasts, for instance, not to mention aerodynamic engineering).

I explicitly wanted to show an example where your assumption that "the other variables are automatically independent of each other" isn't true. More straightforward examples include any variable substitution which doesn't use orthogonal basis. Things get even more interesting when you go into the formalism of differential forms - there, the derivatives become tied to the definition of the underlying space. I'll let others post more about this if needed.

EDIT:

There seems to be a confusion in what exactly the partial derivative does. It takes a function of multiple variables (a function as an object "knows" which term comes from which argument), and only varies one of them. You should write it as such: $f=f(x,t)$. Now, recall the definition:

$$\frac{\partial f}{\partial t}=\lim_{h\to 0}\frac{f(x,t+h)-f(x,t)}{h}$$ Whatever $x$ was, it isn't even touched. So... it doesn't matter if it depends on $t$ or not. It could be $x=t$ and it still wouldn't matter, because you are putting it into the "slot" that isn't varied by the derivative. That's what total derivative is for: in checks the entire effect of varying a variable, not just the argument of a function. Look at it this way: you are given a function. Say, $f(x,t)=x+t^2$. This fixes the partial derivative already. It's $f_t(x,t)=2t$. Imagine this function describes something (let's say temperature) in relation of position and time. The partial derivative tells you how the temperature changes with time at a fixed point. It only depends on the function you have (the physical circumstances). The function knows $x$ is position and $t$ is time, even if the numbers you choose to put into $x$ happen to vary with time. Now, consider what a total derivative does: it checks the actual total effect time has on temperature! You get $df/dt=dx/dt + 2t$. What does that mean? Now this doesn't only depend on the function $f$. It also depends on your trajectory through time and space. If you are stationary (your position doesn't change with time), you get the same answer as before. But if you happen to move (let's say $x=2t$), then the total derivative you are experiencing with time is $df/dt=2+2t$. The total derivative is not a property of a function - it's a property of the function's value in response to a particular trajectory you take in the multiparameter space. It's a completely different creature. The total derivative can be treated as a differential (small displacement) in approximations and physical sketches. The partial derivative is a parameter of the function that tells you something about a neighbourhood of a point. For a scalar function $f(x,y,z,t)$, $df/dx$ is a scalar. $\partial f/\partial x$ is a component of a vector (gradient). Partial derivatives are fields (they exist in all points in parameter space). Total derivatives only make sense, if you happen to be moving in this field. Applying again to physics: partial derivatives of the potential energy tell you the force field (gravity field for gravitational potential energy - the gravitational acceleration is defined in every point in space no matter what you are doing). Total derivative of potential energy tells you about work that you do when you move in this field in a particular direction. If you are going horizontally, you do no work, even though the gravity is still there.

I threw in a bunch of different explanations - hopefully, one of them will make sense.


In terms of programming:

Let's have function $f(x,t)$. Partial derivative takes only this function and creates another function out of it. Python-like pseudocode:

# original function
def f(x,t):
    pass #whatever the function is
# create partial derivative function
def df_dx(x,t):
    return (f(x+h,t)-f(x,t))/h

#now you call this function with arguments of your choice (from now on, it's a function like any other)
t=2.5
x=t*t+0.5 #this dependence on t does nothing for the derivative, the function is already defined by now
value=df_dx(x,t)

The total derivative does something else. You need to know your trajectory in advance, and put it into the function before taking derivative

def f(x,t):
    pass #whatever the function is
#define trajectory, depending on the parameter
def x(t):
    pass #whatever the trajectory is

#define a wrapper function that takes the trajectory into account
def f_(t):
    return f(x(t),t)

#NOW take the derivative
def df(t):
    return (f_(t+h)-f_(t))/h

#compute the value of total derivative - no choice in trajectory anymore, it's hard-coded in the total derivative
value=df(2.5)

Note that I oversimplified here. The parameter of total differentiation isn't necessarily one of the original variables. I could just as easily have defined

def x(u):
    return something
def t(u):
    return something else
def f_(u):
    return f(x(u),t(u))

Clear?

$\endgroup$
  • $\begingroup$ can you explain why my examples wouldn't work though? $\endgroup$ – AlanSTACK Jan 25 '16 at 9:05
  • $\begingroup$ In case 2), your second bullet is just plain wrong. By definition, partial derivative only includes the first term! The line $\partial f/\partial x = \partial f / \partial x + $ no more terms (after all, the first term is already what you had on the left). The partial derivative does not differentiate over other variables on which the function depends! $\endgroup$ – orion Jan 25 '16 at 9:11
  • 1
    $\begingroup$ @gary First of all, you want total derivative $d^2f/dt^2$, not partial (partial is $0$ in this case, as $f$ doesn't depend on $t$ directly, only through $x,y$). And then, you didn't apply the product rule correctly. I'll only demonstrate for $x$ part. $$\frac{d}{dt}\frac{df}{dt}=\frac{d}{dt}\left(\frac{\partial f}{\partial x}\frac{dx}{dt}\right)=\frac{\partial^2 f}{\partial x^2} \left(\frac{dx}{dt}\right)^2+\frac{\partial f}{\partial x}\frac{ d^2x}{dt^2}$$ You forgot the first term: The partial derivative $\partial f/\partial x$ is still dependent on $t$ and you have to take this into account. $\endgroup$ – orion Sep 5 '18 at 6:41
  • 1
    $\begingroup$ Also, when you expand it in two variables, you will get mixed derivatives, too. For example, one of the terms of differentiating the product $\frac{\partial f}{\partial x}\frac{dx}{dt}$ will require this side calculation: $$\frac{d}{dt}\frac{\partial f}{\partial x}=\frac{\partial^2 f}{\partial x^2}\frac{dx}{dt}+\frac{\partial^2 f}{\partial x\partial y}\frac{dy}{dt}$$ The same will happen on $y$ side. This means you will get $6$ terms, but two will be equal because mixed derivative does not depend on the order of differentiation. $\endgroup$ – orion Sep 5 '18 at 6:56
  • 1
    $\begingroup$ The jacobian only contains first derivatives so it will not be enough (second derivatives are usually packed into the Hessian matrix). Also, jacobian is used if you have the same number of inputs and outputs. Here, $f$ is a scalar, and there is only one $t$ which is a parameter of the trajectory. It would be useful if you just wanted to change variables from $x,yt$ to $t,u$. $\endgroup$ – orion Sep 7 '18 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.