1
$\begingroup$

$\frac{d\bf{Ax}}{d\bf{x}'} = \bf{A}$ makes sense as $\bf{Ax}$ becomes a column vector given $\bf{x}$ is a column vector and $\bf{x}'$ is a row vector and $\bf{A}$ is a $n\times n$ matrix, but I see elsewhere that states $\frac{d\bf{Ax}}{d\bf{x}} = \bf{A}'$, which doesn't makes sense (to me!) because it takes a derivative of $\bf{Ax}$ by the column vector which should place each partial column-wise. It only makes sense if somehow $Ax$ is a row vector such as $\bf{x'A'}$.

What is the explanation of this result?

$\endgroup$
2
  • $\begingroup$ There are a lot of conventions for whether the derivative of something w.r.t a row should be a row or a column. Have you checked the wiki article on matrix calculus? $\endgroup$ Apr 29, 2019 at 1:37
  • $\begingroup$ I found things called "Denominator Layout Notation", and if you follow their convention, it makes sense. $\endgroup$
    – Ikuyasu
    Apr 29, 2019 at 2:01

1 Answer 1

3
$\begingroup$

Index notation often brings clarity to this kind of problem. The only thing you need to know is $$\frac{\partial x_i}{\partial x_j} = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i\ne j \end{cases}$$

For the current problem $y=Ax$ we have $$\eqalign{ y_i &= A_{ij}x_j \cr \frac{\partial y_i}{\partial x_k} &= A_{ij}\frac{\partial x_i}{\partial x_k} = A_{ij}\delta_{ik} = A_{ik} \cr \frac{\partial(Ax)}{\partial x} &= A \cr }$$ where summation over a repeated index is implied (aka Einstein convention).

Differentiating $w^T = x^TA^T\,$ produces $$\eqalign{ w_i &= x_jA^T_{ji} = A_{ij}x_j =y_i \cr \frac{\partial w_i}{\partial x_k} &= \frac{\partial y_i}{\partial x_k} = A_{ik} \cr \frac{\partial(x^TA^T)}{\partial x} &= A \cr }$$ You don't need to concern yourself with metric tensors or dual bases unless you're working with a non-Euclidean space (general relativity) or an oblique coordinate system (crystallography).

In standard cartesian coordinates you don't need to distinguish between $y^T$ and $y$. It just happens to be a limitation of matrix notation which creates this confusion.

$\endgroup$
3
  • $\begingroup$ Nevertheless, transposes can be expressed in index notation, so you can actually translate back to matrices if you want. Also this helps one to give meaning to $\frac{\partial x^T}{\partial x}$ and other similar objects. $\endgroup$ Apr 30, 2019 at 2:47
  • $\begingroup$ Can you make it clear which quantities are vectors? When you put subscript on variables, I am not sure if it's a component of vector indexed or an indexed column/row vector. $\endgroup$
    – Ikuyasu
    May 1, 2019 at 1:29
  • $\begingroup$ @Ikuyasu All of the indexed quantities are scalar elements of the corresponding matrix or vector. For example, $x_k$ is the $k^{th}$ component of the vector $x$, while $A_{mn}$ is the element located in the $m^{th}$ row and $n^{th}$ column of the matrix $A$. $\endgroup$
    – greg
    May 1, 2019 at 2:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .