# Derivative of a multivariate quadratic function

Let's define function $$f : \mathbb{R}^n \to \mathbb{R}$$ as

$$f(x) = {1\over2}x'Ax + b'x$$

where matrix $$A \in \mathbb{R}^{n\times n}$$ and vector $$b\in \mathbb{R}^n$$ are given. Function $$f$$ is twice differentiable.

How can one prove the following?

$$\nabla f(x) = {1\over 2} (A+A')x + b$$

• Compute the directional derivative. Then extract the gradient. Take a look at this. Mar 30, 2019 at 9:34

Well you could expand it as $$f(x) = \frac{1}{2} \sum_i \sum_j x_ix_j A_{ij} + \sum_i b_ix_i$$ Derive with respect to $$x_k$$
$$\frac{\partial}{\partial x_k} f(x) =\frac{1}{2}(\sum_j x_j A_{kj} + \sum_i x_i A_{ik} ) + b_k$$
$$\begin{bmatrix} \frac{\partial}{\partial x_1} f(x)\\ \frac{\partial}{\partial x_2} f(x)\\ \vdots\\ \frac{\partial}{\partial x_n} f(x) \end{bmatrix} = \begin{bmatrix} \frac{1}{2}(\sum_j x_j A_{1j} + \sum_i x_i A_{i1} ) + b_1\\ \frac{1}{2}(\sum_j x_j A_{2j} + \sum_i x_i A_{i2} ) + b_2\\ \vdots\\ \frac{1}{2}(\sum_j x_j A_{nj} + \sum_i x_i A_{in} ) + b_n \end{bmatrix}$$ The first term above it $$\frac{1}{2} Ax$$, the second is $$\frac{1}{2} A^T x$$ and the third is $$b$$.
• thank you very much @AhmadBazzi for your answer. In the answer I got from my course I got the following first statement: $$\forall k\leq n: \frac{\partial f}{\partial x_k}(x) =\frac{1}{2} \big[\sum_{i\neq k}x_i a_{ik} + \sum_{j\neq k}a_{kj}x_j + 2a_{kk}x_k\big] + b_k$$ now: where does $2a_{kk}x_k$ come from and why is there both $i\neq k$ and $j\neq k$ under both sum signs?
• it's the same: if you look at each of the summations that I wrote, you have an instance where $j =k$ and $i = k$. In your expression, the $k^{th}$ term is extracted out of each to emphasise that this term is in fact equal and thus sums up to $2a_{kk}x_k$. Mar 30, 2019 at 9:56