1
$\begingroup$

I was going through my linear algebra notes and got a bit confused with the following: let $x$ be a vector in $\mathbb{R}^{n}$ and $A$ an $n\times n$ matrix, then $$ \frac{\partial x'A}{\partial x}=A $$ Since I was confused I tried the following "toy" example. Let $A$ be a $2\times 2$ matrix given by $$ A=\begin{bmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{bmatrix} $$ Thus $$ x'A=\begin{bmatrix}a_{11}x_{1}+a_{21}x_{2}&a_{12}x_{1}+a_{22}x_{2}\end{bmatrix} $$ Now how do I take "the derivative" of each element? Is such thing the Jacobian of the function $f(x_{1},x_{2})=\left(a_{11}x_{1}+a_{21}x_{2},a_{12}x_{1}+a_{22}x_{2}\right)$?

Thank you very much

$\endgroup$
6
  • $\begingroup$ Why is $x^\textsf{T}A$ a $2 \times 1$ matrix? Now, if you take the derivative elementwise, how do you end up with a $2\times2$ matrix? None of this makes sense to me! $\endgroup$
    – azif00
    Aug 30, 2020 at 23:38
  • $\begingroup$ @azifmedrano I made a mistake in the question. Could you check the post again please? $\endgroup$ Aug 30, 2020 at 23:45
  • 1
    $\begingroup$ The function $f(x) = x^T A$ is linear hence $Df(x_0) = f$. One can write $Df(x_0) (h) = h^T A$, this differs slightly from the usual premultiplication by $A$. $\endgroup$
    – copper.hat
    Aug 31, 2020 at 0:06
  • $\begingroup$ @copper.hat Thank you very much, you mean $Df(x_{0})=A$ ? $\endgroup$ Aug 31, 2020 at 0:26
  • $\begingroup$ @PedroIgnacioMartinezBruera No, I meant the function $f$. $Df(x_0)$ is a (linear) function. $\endgroup$
    – copper.hat
    Aug 31, 2020 at 0:29

1 Answer 1

1
$\begingroup$

Allright so I have checked the book "Matrix Algebra From a Statistician's Perspective" by Harville and I think I have reached an answer:

Let $\mathbf{f}=(f_{1},f_{2},\ldots,f_{n})'$ be a (column) vector function and $\mathbf{x}=\left(x_{1},x_{2},\ldots,x_{n}\right)'$. We define $\frac{d\mathbf{f}}{d\mathbf{x}'}$ as the Jacobian of such function, i.e. the $ij_{th}$ element is $\frac{df_{i}}{dx_{j}}$. In addition, we define $\frac{d\mathbf{f}'}{d\mathbf{x}}=\left(\frac{d\mathbf{f}}{d\mathbf{x}'}\right)'$ as the gradient matrix.

Thus, it is trivial to show that $\frac{d A\mathbf{x}}{d\mathbf{x}'}=A$ and therefore $\frac{d\mathbf{x}'A}{d\mathbf{x}}=\left(\frac{dA'\mathbf{x}}{d\mathbf{x}'}\right)'=\left(A'\right)'=A$.

In order to show that $\frac{dA\mathbf{x}}{d\mathbf{x}}=A'$ we know that $\frac{dA\mathbf{x}}{d\mathbf{x}}=\left(\frac{d\mathbf{x}'A'}{d\mathbf{x}'}\right)'$. The problem is that $\mathbf{x}'A$ is not a column of functions! What we need to do is to "redefine" it as the derivative of its transpose -$A\mathbf{x}$- in order to get a column (whose derivative we know how to compute). Since $\frac{dA\mathbf{x}}{d\mathbf{x}'}=A$ we conclude that $\frac{dA\mathbf{x}}{d\mathbf{x}}=A'$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.