# Vector valued function derivative with matrix

Given a function $$f: \mathbb{R}^n \rightarrow \mathbb{R}^n$$ and a matrix $$A \in \mathbb{R}^{n \times n}$$. Is there a general formula for calculating the following derivative:

$$\frac{\partial}{\partial x} f(x)^T A f(x) \tag{1} = ?$$

I know that

$$\frac{\partial}{\partial x} x^T A x = x^T(A + A^T) \overset{A = A^T}{=} 2 x^T A \tag{2}$$

and the solution to $$(1)$$ will probably look similar to $$(2)$$, but I am stuck here since I am not sure how to apply the chain rule in the matrix case.

Edit: Regarding notation, we have

$$\frac{\partial }{\partial x}f(x) = \begin{bmatrix} \frac{\partial}{\partial x_1} f_1(x) & \frac{\partial}{\partial x_2} f_1(x) & \cdots & \frac{\partial}{\partial x_n} f_1(x) \\ \frac{\partial}{\partial x_1} f_2(x) & \frac{\partial}{\partial x_2} f_2(x) & \cdots & \frac{\partial}{\partial x_n} f_2(x) \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial}{\partial x_1} f_n(x) & \frac{\partial}{\partial x_2} f_n(x) & \cdots & \frac{\partial}{\partial x_n} f_n(x) \end{bmatrix}$$

and

$$x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} , f(x) = \begin{bmatrix} f_1(x) \\ f_2(x) \\ \vdots \\ f_n(x) \end{bmatrix}$$

• Is $\partial/\partial x$ the total differential (or Jacobian, however you want to call it)? The expression $f(x)^TAf(x)$ is a product of things, so you can appeal to the product rule (either a general product rule, or do it entry wise). – Reveillark Feb 26 '19 at 20:56
• @Reveillark I updated the question. I know I could do everything elementwise using the product rule, but I am rather looking for a compact formula in matrix notation, similar to $(2)$. – SampleTime Feb 26 '19 at 21:25

Given a differentiable vector field $$\mathrm v : \mathbb R^n \to \mathbb R^n$$ and a matrix $$\mathrm A \in \mathbb R^{n \times n}$$, let function $$f : \mathbb R^n \to \mathbb R$$ be defined by

$$f (\mathrm x) := \langle \mathrm v (\mathrm x), \mathrm A \mathrm v (\mathrm x) \rangle$$

whose directional derivative in the direction of $$\mathrm y \in \mathbb R^n$$ at $$\mathrm x \in \mathbb R^n$$ is

$$D_{\mathrm y} f (\mathrm x) := \lim_{h \to 0} \frac{f (\mathrm x + h \mathrm y) - f (\mathrm x)}{h} = \cdots = \langle \mathrm y, \mathrm J_{\mathrm v}^\top (\mathrm x) \, \mathrm A \, \mathrm v (\mathrm x) \rangle + \langle \mathrm J_{\mathrm v}^\top (\mathrm x) \, \mathrm A^\top \mathrm v (\mathrm x) , \mathrm y \rangle$$

where matrix $$\mathrm J_{\mathrm v} (\mathrm x)$$ is the Jacobian of vector field $$\rm v$$ at $$\mathrm x \in \mathbb R^n$$. Thus, the gradient of $$f$$ is

$$\nabla_{\mathrm x} f (\mathrm x) = \mathrm J_{\mathrm v}^\top (\mathrm x) \left( \mathrm A + \mathrm A^\top \right) \mathrm v (\mathrm x)$$

I find differential notation helpful here in organizing things. The total derivative is a linear operator, so we introduce its argument and apply the product rule: $$d(f(x)^TAf(x))=d(f(x)^T)\cdot Af(x)+f(x)^TA\cdot d(f(x)$$ $$d(f(x)^TAf(x))=\left(\frac{df}{dx}\cdot dx\right)^T\cdot Af(x)+f(x)^TA\cdot \left(\frac{df}{dx}\cdot dx\right)$$ Now, the transpose of a $$1\times 1$$ matrix is itself, so we transpose that first term: $$d(f(x)^TAf(x)) = f(x)^TA^T\cdot \left(\frac{df}{dx}\cdot dx\right)+f(x)^TA\cdot \left(\frac{df}{dx}\cdot dx\right)$$ $$d(f(x)^TAf(x)) = f(x)^T(A+A^T)\cdot \left(\frac{df}{dx}\cdot dx\right)$$ Now that's in the form we want for the derivative. The total derivative of $$f(x)^TAf(x)$$ is $$f(x)^T(A+A^T)\frac{df}{dx}$$ where $$\frac{df}{dx}$$ is the matrix of partial derivatives of $$f$$, written in your question as $$\frac{\partial}{\partial x}f(x)$$.

Well we want to differentiate $$f(x)^TAf(x)$$ then it is useful to break into pieces.

First we see how to differentiate $$g(x,y) = x^TAy$$ with $$A$$ constant. $$g(x+h,y+k) = (x+h)^TA(y+k) =(x^T+h^T)A(y+k) = x^TAy + h^TAy + x^TAk + h^TAk$$ From this we see that $$Dg_{(x,y)}(h,k) = h^TAy + x^TAk$$.

Now we use the chain rule $$D(f(x)^TAf(x))_x(v) = Dg_{(f(x),f(x))}(Df_x(v),Df_x(v)) = Df_x(v)^TAf(x) + f(x)^TADf_x(v)$$