0
$\begingroup$

I know how to differentiate a product of matrices wrt a given matrix-entry e.g., $X_{i, j}$. However, I'm not sure if I'm thinking correctly about how to do that when I have a vector i.e., just one column.

I have $\mathbf{x^{\top}Ax}$ and I want it's derivative wrt $x_{i}$. Suppose that $\mathbf{x}$ is $4 \times 1$ and that $\mathbf{A}$ is $4 \times 4$ and symmetric! What follows is correct?

$$ \frac{\partial \mathbf{x^{\top}Ax}}{\partial x_{1}} = 2\mathbf{J}^{1\cdot}\mathbf{Ax} = 2 \begin{bmatrix} 1 & 1 & 1 & 1\\ 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 \end{bmatrix} \mathbf{Ax}. $$

Tks in advance!

$\endgroup$
  • $\begingroup$ how did you compute that expression? $\endgroup$ – Exodd Apr 7 at 20:26
  • $\begingroup$ Why is your partial derivative a $4 \times 1$ vector? $\frac{\partial x^TAx}{\partial x_1}$ should be a scalar $\endgroup$ – Ben Grossmann Apr 7 at 20:27
  • $\begingroup$ So the answer is no: your answer is not correct. $\endgroup$ – Ben Grossmann Apr 7 at 20:28
  • $\begingroup$ We know that $\partial \mathbf{x^{\top}Ax}/\partial \mathbf{x} = 2\mathbf{Ax}$. When we deal with $\mathbf{X}$ instead of $\mathbf{x}$ and differentiate wrt $X_{ij}$, we introduce the single-entry matrix $\mathbf{J}^{ij}$. What I did here was to 'adapt' the idea of the single-entry matrix to the context of a vector $\mathbf{x}$. @Omnomnomnom, you're saying that since I'm differentiating in one value, my output should be a scalar, not a vector - even with the other elements of that vector being zero (as I proposed), right? What you said makes sense to me, but then I don't know how to solve it. $\endgroup$ – Henrique Laureano Apr 7 at 20:56
  • 1
    $\begingroup$ Okay, you know the gradient is $2Ax$, so if you multiply by the $k^{th}$ basis vector you'll get $$2e_k^TAx = e_k^T\left(\frac{\partial f}{\partial x}\right) = \frac{\partial f}{\partial x_k}$$ Think of $e_k$ as the single-entry vector. $\endgroup$ – greg Apr 7 at 21:13
1
$\begingroup$

The answer is simple, Greg explained clearly in the comments section.

We know that $$ \frac{\partial \mathbf{x}^{\top}\mathbf{Ax}}{\partial \mathbf{x}} = 2\mathbf{Ax}. $$ So, if now I want the derivative wrt $x_{k}$ I just multiply it by the $k^{\text{th}}$ basis vector. i.e., $$ \frac{\partial \mathbf{x}^{\top}\mathbf{Ax}}{\partial x_{k}} = 2e_{k}^{\top}\mathbf{Ax} = e_{k}^{\top} \frac{\partial \mathbf{x}^{\top}\mathbf{Ax}}{\partial \mathbf{x}}. $$

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

I assume that $A$ is a matrix having real numbers as entries. Hence with $x^T:=(x_1,\ldots,x_n)$ the term $x^TAx$ is a quadratic polynomial in $n$ variables $x_1,\ldots,x_n$ which you can differentiate w.r.t. $x_1$.

To your calculation: That equation does not hold, since $x^TAx$ is a real number for all $x_1,\ldots,x_n$ and not $4\times 1$ as in the middle and rhs.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ If I have $\mathbf{x}^{\top} = [x_{1} x_{2}]$ and $\mathbf{A}$ with entries, $a, b, c, d$, $\mathbf{x}^{\top}\mathbf{Ax} = ax_{1}^{2} + (b+c)x_{1}x_{2} + dx_{2}^{2}$. Are you saying that the derivative of this polynomial wrt $x_{1}$ is my answer? Ok, but then how I generalize it in a matrix form? $\endgroup$ – Henrique Laureano Apr 7 at 21:09
  • $\begingroup$ If $A$ is a $n\times n$ matrix and also $x$ is a $n\times k$ matrix with monomials as entries then you can compute the matrix product and derivate the result entrywise. Since the set of all $l\times k$ matrices $Mat(l,k;\mathbb{R})$ is a vector space you are able to define a notion of differentiability for maps $A: \mathbb{R}\longrightarrow Mat(l,k;\mathbb{R})$, which you would have in that case. And yes, that was what I proposed. Just the partial differential of $ax^2_1+(b+c)x_1x_2+dx_2^2$ $\endgroup$ – dennis_s Apr 8 at 10:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.