3
$\begingroup$

Let's define function $f : \mathbb{R}^n \to \mathbb{R}$ as

$$ f(x) = {1\over2}x'Ax + b'x $$

where matrix $A \in \mathbb{R}^{n\times n}$ and vector $b\in \mathbb{R}^n$ are given. Function $f$ is twice differentiable.

How can one prove the following?

$$\nabla f(x) = {1\over 2} (A+A')x + b$$

$\endgroup$
1
  • $\begingroup$ Compute the directional derivative. Then extract the gradient. Take a look at this. $\endgroup$
    – Rodrigo de Azevedo
    Commented Mar 30, 2019 at 9:34

1 Answer 1

Reset to default
1
$\begingroup$

Well you could expand it as $$f(x) = \frac{1}{2} \sum_i \sum_j x_ix_j A_{ij} + \sum_i b_ix_i$$ Derive with respect to $x_k$

$$\frac{\partial}{\partial x_k} f(x) =\frac{1}{2}(\sum_j x_j A_{kj} + \sum_i x_i A_{ik} ) + b_k$$

This means that

$$ \begin{bmatrix} \frac{\partial}{\partial x_1} f(x)\\ \frac{\partial}{\partial x_2} f(x)\\ \vdots\\ \frac{\partial}{\partial x_n} f(x) \end{bmatrix} = \begin{bmatrix} \frac{1}{2}(\sum_j x_j A_{1j} + \sum_i x_i A_{i1} ) + b_1\\ \frac{1}{2}(\sum_j x_j A_{2j} + \sum_i x_i A_{i2} ) + b_2\\ \vdots\\ \frac{1}{2}(\sum_j x_j A_{nj} + \sum_i x_i A_{in} ) + b_n \end{bmatrix}$$ The first term above it $\frac{1}{2} Ax$, the second is $\frac{1}{2} A^T x$ and the third is $b$.

$\endgroup$
2
  • $\begingroup$ thank you very much @AhmadBazzi for your answer. In the answer I got from my course I got the following first statement: $$ \forall k\leq n: \frac{\partial f}{\partial x_k}(x) =\frac{1}{2} \big[\sum_{i\neq k}x_i a_{ik} + \sum_{j\neq k}a_{kj}x_j + 2a_{kk}x_k\big] + b_k $$ now: where does $2a_{kk}x_k$ come from and why is there both $i\neq k$ and $j\neq k$ under both sum signs? $\endgroup$
    – ecjb
    Commented Mar 30, 2019 at 9:53
  • $\begingroup$ it's the same: if you look at each of the summations that I wrote, you have an instance where $j =k $ and $i = k$. In your expression, the $k^{th}$ term is extracted out of each to emphasise that this term is in fact equal and thus sums up to $2a_{kk}x_k$. $\endgroup$
    – Ahmad Bazzi
    Commented Mar 30, 2019 at 9:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .