What's gone wrong in my example to show that $\frac{\partial}{\partial \underline c}(A\underline c) = A$?

First, an example:

Suppose $$A$$ is a symmetric matrix $$A = \begin{bmatrix} a & b \\ b & a \\ \end{bmatrix}$$, and $$\underline c = \begin{bmatrix} c_1 \\ c_2 \end{bmatrix}$$

Then $$A \underline c = \begin{bmatrix} ac_1 + b c_2 \\ bc_1 + ac_2 \\ \end{bmatrix}$$

and hence, $$\frac{\partial}{\partial \underline c}(A\underline c) = \begin{bmatrix} \frac{\partial}{\partial c_1}(ac_1 + b c_2) \\ \frac{\partial}{\partial c_2}(bc_1 + ac_2) \\ \end{bmatrix} = \begin{bmatrix} a \\a \end{bmatrix} \neq A$$

However, I'm told that the derivative with respect to the vector of the product of a symmetric $$(n\times n)$$ matrix $$A$$ and an $$(n\times 1)$$ vector $$\underline c$$ is equal to the matrix $$A$$, such that,

$$\frac{\partial}{\partial \underline c}(A\underline c) = A$$

Where is the error in my understanding?

• Others have answered the question, but I thought I should point out that your $A$ is not the most general $2\times 2$ symmetric matrix. The two terms on the diagonal, both $a$ in your example, can be distinct. Jul 23, 2019 at 13:58

You have a vector result $$y = Ac$$

Thus, you are computing a vector-by-vector gradient which results in a second-order tensor, ie. a matrix, with the following form:

$$\frac{\partial y}{\partial c}= \begin{bmatrix} {\frac {\partial y_{1}}{\partial c_{1}}}&{\frac {\partial y_{1}}{\partial c_{2}}}\\ {\frac {\partial y_{2}}{\partial c_{1}}}&{\frac {\partial y_{2}}{\partial c_{2}}}\\ \end{bmatrix}$$

You are just missing the off-diagonal terms.

Since your function is vector valued, its derivative is a matrix : $$\left(\frac{\partial \mathbf{u}}{\partial\mathbf{x}}\right)_{i,j} = \frac{\partial u_i}{\partial x_j}$$.

Hence, $$\frac{\partial A \mathbf{c}}{\partial \mathbf{c}} = \begin{bmatrix} \frac{\partial a c_1 + b c_2}{\partial c_1} & \frac{\partial a c_1 + b c_2}{\partial c_2} \\ \frac{\partial b c_1 + a c_2}{\partial c_1} & \frac{\partial b c_1 + a c_2}{\partial c_2}\end{bmatrix} = \begin{bmatrix} a & b\\ b & a\end{bmatrix}$$.

Remark : in general, $$\frac{\partial A \mathbf{c}}{\partial \mathbf{c}} = A^T$$, but here your matrix is symmetric (i.e. $$A=A^T$$).

Your definition of the derivative $$\frac{\partial}{\partial \overline{c}}.$$

The function $$\overline{c} \mapsto A \overline{c}$$ is total differentiable if there exists a linear map $$B$$, with $$\lim_{\|h\| \rightarrow 0} \frac{\|A\left(\overline{c}+\overline{h}\right) - A\overline{c} - B\overline{h}\|}{\|\overline{h}\|} = 0$$.

Since the map is linear, it is obvious that $$A = B$$, so the total derivative is $$A$$.

• I'm computing a little and getting \begin{align} ... = \lim_{||h|| \rightarrow 0} \frac{|| -A\bar{h}-B\bar{h} ||}{||\bar{h}||}=0 \end{align} can you explain how this requires $A=B$? Jul 23, 2019 at 13:43
• Yes sorry, I accidentally exchanged the $A\left(\overline{c} + \overline{h}\right)$ and the $A \overline{c}$ terms... Edited it now to bring it into the right order. Jul 23, 2019 at 13:49