Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $x \in \mathbb{R}^n$

What is

$$\frac{\partial}{\partial x} [ x^Tx ]$$

My guess is: $\frac{\partial}{\partial x} [ x^Tx ] = 0$, because $[x^Tx] \in \mathbb{R}^1$, hence a real number as is interpreted as scalar in this derivation.

share|cite|improve this question
i think norm of any vector is always a positive real number so it is constant and derivative of constant function is zero. – Kns Apr 16 '12 at 9:16
@Kunjanshah: Do you think every function $\Bbb R^n\to \Bbb R$ is constant? – anon Apr 16 '12 at 9:25
up vote 3 down vote accepted

Let $u:\mathbb R^n\to\mathbb R$, $x\mapsto u(x)=x^Tx$. There exists a linear application $\ell_x:\mathbb R^n\to\mathbb R$, called the gradient of $u$ at $x$, such that $u(x+z)=u(x)+\ell_x(z)+o(\|z\|)$ when $z\to0$. To compute $\ell_x$, note that $$ u(x+z)=(x+z)^T(x+z)=x^Tx+z^Tx+x^Tz+z^Tz=x^Tx+2x^Tz+o(\|z\|), $$ hence $$ \ell_x(z)=2x^Tz. $$ Every linear form $\ell$ on $\mathbb R^n$ has the form $\ell:z\mapsto w^Tz$ for some $w$ in $\mathbb R^n$ hence one often identifies $\ell$ with $w$ (technically, this is identifying the dual of $\mathbb R^n$ with $\mathbb R^n$). In the present case, one identifies the gradient $\ell_x$ of $u$ at $x$ ( a linear application from $\mathbb R^n$ to $\mathbb R$) with the vector $2x$ (an element of $\mathbb R^n$), and one often writes $$ (\text{grad}\ u)(x)=2x. $$

share|cite|improve this answer
Thanks for this specific answer, but I am afraid this does not help me. Is this a counter example, why do you introduce new variables? What is $\frac{\partial}{\partial x} [x^Tx]$ after all? – Mahoni Apr 16 '12 at 9:40
Nothing specific here, please read again: the object you call $\frac{\partial}{\partial x}(x^Tx)$ (using a notation I cannot recommend) is $(\text{grad}\ u)(x)$, that is, $2x$. – Did Apr 16 '12 at 9:45
This clarifies things a lot and what about $(grad\ x) (x^T x)$? – Mahoni Apr 16 '12 at 9:53
Not a valid expression. – Did Apr 16 '12 at 9:57

Write x as $(x_1, x_2, \cdots, x_n)$. Then $x^t x = \sum_i x_i^2$. So, for example, $$\frac{d}{dx_1} x^t x = \frac{d}{dx_1} \left( \sum x_i^2\right) = \frac{d}{dx_1} x_1^2 = 2x_1$$ and similarly for each of the other components of $x$. From this it should be clear that $$\frac{d}{dx} x^t x = 2x^t$$ (The transpose is there because the derivative is a map $\mathbb{R}^n\rightarrow\mathbb{R}$, so expressed as a matrix it must have dimension $1\times n$, or alternatively, as a linear map it must live in the dual space to $\mathbb{R}^n$, i.e. the space of linear maps $\mathbb{R}^n \rightarrow \mathbb{R}$.)

Your question perhaps betrays some confusion as to what the derivative is. Although for each $x$ the value of $x^t x$ is a single number, i.e. a scalar, the derivative expresses the amount by which $x^t x$ changes as the entries of $x$ change. This is surely nonzero, since the value of $x^t x$ depends on the entries of $x$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.