2
$\begingroup$

Given a function $f: \mathbb{R}^n \rightarrow \mathbb{R}^n$ and a matrix $A \in \mathbb{R}^{n \times n}$. Is there a general formula for calculating the following derivative:

$$ \frac{\partial}{\partial x} f(x)^T A f(x) \tag{1} = ? $$

I know that

$$ \frac{\partial}{\partial x} x^T A x = x^T(A + A^T) \overset{A = A^T}{=} 2 x^T A \tag{2} $$

and the solution to $(1)$ will probably look similar to $(2)$, but I am stuck here since I am not sure how to apply the chain rule in the matrix case.

Edit: Regarding notation, we have

$$ \frac{\partial }{\partial x}f(x) = \begin{bmatrix} \frac{\partial}{\partial x_1} f_1(x) & \frac{\partial}{\partial x_2} f_1(x) & \cdots & \frac{\partial}{\partial x_n} f_1(x) \\ \frac{\partial}{\partial x_1} f_2(x) & \frac{\partial}{\partial x_2} f_2(x) & \cdots & \frac{\partial}{\partial x_n} f_2(x) \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial}{\partial x_1} f_n(x) & \frac{\partial}{\partial x_2} f_n(x) & \cdots & \frac{\partial}{\partial x_n} f_n(x) \end{bmatrix} $$

and

$$ x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} , f(x) = \begin{bmatrix} f_1(x) \\ f_2(x) \\ \vdots \\ f_n(x) \end{bmatrix} $$

$\endgroup$
  • $\begingroup$ Is $\partial/\partial x$ the total differential (or Jacobian, however you want to call it)? The expression $f(x)^TAf(x)$ is a product of things, so you can appeal to the product rule (either a general product rule, or do it entry wise). $\endgroup$ – Reveillark Feb 26 at 20:56
  • $\begingroup$ @Reveillark I updated the question. I know I could do everything elementwise using the product rule, but I am rather looking for a compact formula in matrix notation, similar to $(2)$. $\endgroup$ – SampleTime Feb 26 at 21:25
2
$\begingroup$

Given a differentiable vector field $\mathrm v : \mathbb R^n \to \mathbb R^n$ and a matrix $\mathrm A \in \mathbb R^{n \times n}$, let function $f : \mathbb R^n \to \mathbb R$ be defined by

$$f (\mathrm x) := \langle \mathrm v (\mathrm x), \mathrm A \mathrm v (\mathrm x) \rangle$$

whose directional derivative in the direction of $\mathrm y \in \mathbb R^n$ at $\mathrm x \in \mathbb R^n$ is

$$D_{\mathrm y} f (\mathrm x) := \lim_{h \to 0} \frac{f (\mathrm x + h \mathrm y) - f (\mathrm x)}{h} = \cdots = \langle \mathrm y, \mathrm J_{\mathrm v}^\top (\mathrm x) \, \mathrm A \, \mathrm v (\mathrm x) \rangle + \langle \mathrm J_{\mathrm v}^\top (\mathrm x) \, \mathrm A^\top \mathrm v (\mathrm x) , \mathrm y \rangle$$

where matrix $\mathrm J_{\mathrm v} (\mathrm x)$ is the Jacobian of vector field $\rm v$ at $\mathrm x \in \mathbb R^n$. Thus, the gradient of $f$ is

$$\nabla_{\mathrm x} f (\mathrm x) = \mathrm J_{\mathrm v}^\top (\mathrm x) \left( \mathrm A + \mathrm A^\top \right) \mathrm v (\mathrm x)$$

$\endgroup$
2
$\begingroup$

I find differential notation helpful here in organizing things. The total derivative is a linear operator, so we introduce its argument and apply the product rule: $$d(f(x)^TAf(x))=d(f(x)^T)\cdot Af(x)+f(x)^TA\cdot d(f(x)$$ $$d(f(x)^TAf(x))=\left(\frac{df}{dx}\cdot dx\right)^T\cdot Af(x)+f(x)^TA\cdot \left(\frac{df}{dx}\cdot dx\right)$$ Now, the transpose of a $1\times 1$ matrix is itself, so we transpose that first term: $$d(f(x)^TAf(x)) = f(x)^TA^T\cdot \left(\frac{df}{dx}\cdot dx\right)+f(x)^TA\cdot \left(\frac{df}{dx}\cdot dx\right)$$ $$d(f(x)^TAf(x)) = f(x)^T(A+A^T)\cdot \left(\frac{df}{dx}\cdot dx\right)$$ Now that's in the form we want for the derivative. The total derivative of $f(x)^TAf(x)$ is $$f(x)^T(A+A^T)\frac{df}{dx}$$ where $\frac{df}{dx}$ is the matrix of partial derivatives of $f$, written in your question as $\frac{\partial}{\partial x}f(x)$.

$\endgroup$
1
$\begingroup$

Well we want to differentiate $f(x)^TAf(x)$ then it is useful to break into pieces.

First we see how to differentiate $g(x,y) = x^TAy$ with $A$ constant. $$ g(x+h,y+k) = (x+h)^TA(y+k) =(x^T+h^T)A(y+k) = x^TAy + h^TAy + x^TAk + h^TAk $$ From this we see that $Dg_{(x,y)}(h,k) = h^TAy + x^TAk$.

Now we use the chain rule $$ D(f(x)^TAf(x))_x(v) = Dg_{(f(x),f(x))}(Df_x(v),Df_x(v)) = Df_x(v)^TAf(x) + f(x)^TADf_x(v) $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.