1
$\begingroup$

$\frac{d(x^Ta)}{dx} = \frac{d(a^Tx)}{dx} = a^T$

I was confused by this simple formula for a few weeks.

I thought $x^Ta$ is an scalar, and it's derivative respect to a column vector should be an vector, e.g. $a$ instead of $a^T$.

am I missing something?

Thank you !!

$\endgroup$
7
  • $\begingroup$ I think it has to do with the fact that the scalar product is defined as $\square \cdot \triangle = \square^\intercal \triangle$ $\endgroup$ Jan 5, 2016 at 16:53
  • $\begingroup$ If $ f:\mathbb R^n \to \mathbb R^m $, then $ f'(x)$ is an $ m \times n$ matrix. $\endgroup$
    – littleO
    Jan 5, 2016 at 17:45
  • $\begingroup$ @ja72 I notice one paradox: could you please explain how this happens? Thank you! $\mathbf{a^T} \frac{d\mathbf{x}}{d\mathbf{x}} = \mathbf{a^T}$ \\ $ \frac{d\mathbf{x^T}}{d\mathbf{x}} \mathbf{a} = \mathbf{a} $ $\endgroup$
    – Long
    Jan 6, 2016 at 9:59
  • $\begingroup$ @littleO you mean the Jacobian Matrix? why it's a row vector when the gradient is a column vector? Thank you ! $\endgroup$
    – Long
    Jan 6, 2016 at 10:02
  • $\begingroup$ @Long replace all $\square^\intercal$ with $\square \cdot$ and see if it makes sense. The transpose is part of an inner product in vector space. $\endgroup$ Jan 6, 2016 at 13:37

1 Answer 1

2
$\begingroup$

Here's the definition of the derivative of a function $y: \Bbb R^n \to \Bbb R$ wrt the column vector $\mathbf x$. $$\frac{dy(\mathbf x)}{d\mathbf x} := \pmatrix{\frac{\partial y(\mathbf x)}{\partial x_1}, \dots, \frac{\partial y(\mathbf x)}{\partial x_n}}$$ Notice that this is a row vector (by definition).

$y(\mathbf x) = \mathbf a^T\mathbf x$ is a scalar function so the result of differentiating it wrt $\mathbf x$ must be some row vector. Now I assume you can prove that $\frac{d\mathbf a^T\mathbf x}{d\mathbf x} = \mathbf a^T = \frac{d\mathbf x^T\mathbf a}{d\mathbf x}$ in this case?

$\endgroup$
2
  • $\begingroup$ Thank you so much! I have one more question about this definition. what was the gradient of this y(x) w.r.t x? was it a column vector? $\endgroup$
    – Long
    Jan 6, 2016 at 9:54
  • $\begingroup$ It has the same components. But whether it's represented by a row or column vector is not entirely standardized. Different authors choose one or the other. $\endgroup$
    – user137731
    Jan 6, 2016 at 11:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .