# Derivative of a vector with respect to a scalar

I'm reading this paper: https://explained.ai/matrix-calculus/index.html#sec4.3 , under the section- Derivatives Involving Scalar Expansion the following is written, and at the end of this section is the equation I've written here:

"$$\vec{f}(\vec{x})=\vec{x}$$ and $$\vec{g}(z)= \vec{1} z$$. (The notation represents a vector of ones of appropriate length.) $$z$$ is any scalar that doesn't depend on $$x$$, which is useful because then $$\dfrac{\partial{z}}{\partial{x_i}}=0$$ for any $$x_i$$ and that will simplify our partial derivative computations. (It's okay to think of variable z as a constant for our discussion here)."

$$\dfrac{\partial(f_i(x_i)\circ g_i(z))}{\partial z}=x_i\dfrac{\partial z}{\partial z}+z\dfrac{\partial x_i}{\partial z}=x_i+?$$

What's the $$?$$, I think it's undefined as derivative with respect to a constant is undefined, but the paper says it's $$0$$, how so?

Any help is highly appreciated.

Well, if you consider $$x_i\frac{\partial z}{\partial z}+z\frac{\partial x_i}{\partial z}$$, we can immediately see that $$\frac{\partial z}{\partial z}=1$$, since intuitively, z changes at the same rate as itself. Symbolically, we could write that $$\frac{\partial x_k}{\partial x_l}=\delta^k_l$$, where $$\delta^k_l=1$$ when $$k=l$$ and $$0$$ otherwise.
Now, it was explicitly stated that $$z$$ does not depend on $$x$$, therefore $$\frac{\partial z}{\partial x_i}=0$$. As $$z$$ does not depend on $$x_i$$, the converse is also true; namely, $$x_i$$ does not depend on $$z$$, since else you could express $$z$$ in terms of $$x_i$$, which would contradict the initial assumption. Hence, $$\frac{\partial x_i}{\partial z}=0$$. This implies that regardless of the value of $$z$$, the second term is zero.
• I had the same direction of thinking, intuitively that seems correct as well, but z is a constant and $\dfrac{\partial{x}}{\partial{z}}$ means changing a variable with respect to a constant, shouldn't that be undefined? $\dfrac{\partial{x}}{\partial{y}}=\dfrac{1}{\dfrac{\partial{y}}{\partial{x}}}$ therefore, since $\dfrac{\partial z}{\partial x_i}=0 \implies \dfrac{\partial{x_i}}{\partial{z}}= \dfrac{1}{\dfrac{\partial{z}}{\partial{x_i}}}$ is undefined, this step may be wrong so I would really appreciate if you correct me here if I'm wrong. Jun 9, 2019 at 8:33
• It is stated in the lines before the equation that $z$ is a scalar parameter, though, which changes matters quite a bit. I agree that taking the derivative of something with respect to a scalar is ill-defined, but taking a derivative with respect to a scalar parameter is perfectly well-defined. Think about the regular derivative in elementary calculus $\frac{df}{dx}$. In this case, $x$ can be regarded as a scalar parameter, and $\frac{df}{dx}$ is well-defined. Therefore, you are not taking the derivative with respect to a "constant" as you stated. Jun 9, 2019 at 23:27