Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My general question is on what to do with a scalar result from a partial derivative.

Suppose a column vector $x$, and a function $f$ which accepts $x$ and returns a scalar. (e.g. proposition 8). Then suppose that in the process of finding $\frac{\partial f }{\partial x} $, that you take the partial derivative with respect to $x_k$ to get the following, $$\frac{\partial f }{\partial x_k} = 2x_k $$

so then, what is the derivative with respect to the entire vector $x$ (keeping in mind this is a hypothetical example and not the same as prop. 8)? I am not sure whether the the following result would be correct, and whether the result should a row or column vector. $$\frac{\partial f }{\partial x} = 2x^T $$

What if the function $f$ here returns a vector or matrix, then is the answer any different here?

share|cite|improve this question
So $\alpha$ is a function of $x$? Or of $x$ and additional variables? – Alex Becker Jul 9 '12 at 2:25
If $\alpha$ is real valued, then you are right. – Ashok Jul 9 '12 at 5:54
I assume $\alpha(\cdot)$ is a function of $x$; but it is unclear whether $\partial\alpha/\partial x=2x_k$ holds just for one $k$ or for all $k$. – Christian Blatter Jul 9 '12 at 7:07
@ChristianBlatter here I mean for $\partial f / \partial x_k$ to hold for any $x_k$ in the vector, and also renamed it to $f$ since I now realize $\alpha$ by convention is a scalar constant instead of a function per se. – T. Webster Jul 10 '12 at 2:31

It's depends on your definition, but usually the derivative of a scalar with respect to a vector is defined as $\frac{\partial \alpha}{\partial\mathbf{x}} \equiv [\frac{\partial \alpha}{\partial x_1},\cdots, \frac{\partial \alpha}{\partial x_n}]^T $ where $\mathbf{x}=[x_1, \cdots, x_n]^T$.

For your question, since $\mathbf{x}$ is a column vector and $\frac{\partial \alpha}{\partial x_k} = 2x_k$ holds for all $k$, then $\frac{\partial \alpha}{\partial\mathbf{x}} = 2\mathbf{x}$, i.e. it's still a column vector by the above definition. (Of course, it can be a row vector by another definition.)

If $\alpha$ is a vector or matrix, then the result is a matrix or three-rank tensor, respectively. You can find more information at here and here or google "matrix calculus".

share|cite|improve this answer
for the first $\alpha/x_k$ it is just differentiating w.r.t. $x_k$, and for $\alpha/x$ it's just supposed to be w.r.t. $x$. – T. Webster Jul 9 '12 at 21:20
@T. Webster, is it a question or just a comment? – chaohuang Jul 9 '12 at 21:25
Just edited the answer according to your question modification. – chaohuang Jul 9 '12 at 21:48
@chohuang, so what I don't understand is why at (45) of proposition (8) (cf URL), we go from a scalar expression $\partial \alpha/\partial x_k$, to an expression (46) $x^T$ instead of the $n \times 1$ vector $x$ – T. Webster Jul 10 '12 at 2:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.