# partial derivative of entry $x_k$ in vector $x$

My general question is on what to do with a scalar result from a partial derivative.

Suppose a column vector $x$, and a function $f$ which accepts $x$ and returns a scalar. (e.g. proposition 8). Then suppose that in the process of finding $\frac{\partial f }{\partial x}$, that you take the partial derivative with respect to $x_k$ to get the following, $$\frac{\partial f }{\partial x_k} = 2x_k$$

so then, what is the derivative with respect to the entire vector $x$ (keeping in mind this is a hypothetical example and not the same as prop. 8)? I am not sure whether the the following result would be correct, and whether the result should a row or column vector. $$\frac{\partial f }{\partial x} = 2x^T$$

What if the function $f$ here returns a vector or matrix, then is the answer any different here?

-
So $\alpha$ is a function of $x$? Or of $x$ and additional variables? – Alex Becker Jul 9 '12 at 2:25
If $\alpha$ is real valued, then you are right. – Ashok Jul 9 '12 at 5:54
I assume $\alpha(\cdot)$ is a function of $x$; but it is unclear whether $\partial\alpha/\partial x=2x_k$ holds just for one $k$ or for all $k$. – Christian Blatter Jul 9 '12 at 7:07
@ChristianBlatter here I mean for $\partial f / \partial x_k$ to hold for any $x_k$ in the vector, and also renamed it to $f$ since I now realize $\alpha$ by convention is a scalar constant instead of a function per se. – T. Webster Jul 10 '12 at 2:31

It's depends on your definition, but usually the derivative of a scalar with respect to a vector is defined as $\frac{\partial \alpha}{\partial\mathbf{x}} \equiv [\frac{\partial \alpha}{\partial x_1},\cdots, \frac{\partial \alpha}{\partial x_n}]^T$ where $\mathbf{x}=[x_1, \cdots, x_n]^T$.
For your question, since $\mathbf{x}$ is a column vector and $\frac{\partial \alpha}{\partial x_k} = 2x_k$ holds for all $k$, then $\frac{\partial \alpha}{\partial\mathbf{x}} = 2\mathbf{x}$, i.e. it's still a column vector by the above definition. (Of course, it can be a row vector by another definition.)
If $\alpha$ is a vector or matrix, then the result is a matrix or three-rank tensor, respectively. You can find more information at here and here or google "matrix calculus".
for the first $\alpha/x_k$ it is just differentiating w.r.t. $x_k$, and for $\alpha/x$ it's just supposed to be w.r.t. $x$. – T. Webster Jul 9 '12 at 21:20
@chohuang, so what I don't understand is why at (45) of proposition (8) (cf URL), we go from a scalar expression $\partial \alpha/\partial x_k$, to an expression (46) $x^T$ instead of the $n \times 1$ vector $x$ – T. Webster Jul 10 '12 at 2:54