The differential $\frac{\delta}{\delta w} w^T \Phi^T \Phi w = 2 \Phi^T \Phi w $ Let $w$ be a $n \times 1$ vector and $\Phi$ be a $n \times n$ matrix. Then the following differentiation rule holds:
$\frac{\delta}{\delta w} w^T \Phi^T \Phi w = 2 \Phi^T \Phi w $
Is there some general principle of vector differentiation that implies this rule? I can verify the rule by writing out the whole matrix product and collecting the terms, but I get no intuitive understanding from doing this. The rule seems very similar to the single variable derivative rule $\frac{d}{dx} ax^2 = 2ax$, which comes from the more general rule $\frac{d}{dx} x^n = nx^{n-1}$. Is there an analogous general rule for vector functions? Is there a way to "just see" the solution without writing out the matrix products?
 A: For several cases, you can follow the steps described in the books:
a) Matrix Differential Calculus with Applications in Statistics and Econometrics (W.E. Shewhart and S. S. Wilks);
b) Complex-Valued Matrix Derivatives (Are Hjørungnes).
For a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$
1) Calculate  $\Delta f = f(\mathbf{x}+\mathbf{\Delta x})-f(\mathbf{x})$;
2) Keep only the first order terms (in $\Delta \mathbf{x}$) of $\Delta f$. Here, you will have been constructing the differential of $f$, namely, $∂f$. 
3) Change, thus, $\Delta \mathbf{x}$ by $\partial \mathbf{x}$ and you'll have $\partial f = (\cdot)\partial \mathbf{x}$. The quantity $(\cdot)$ is the derivative $\frac{\partial f}{\partial\mathbf{x}}$.
For your example, $f(\mathbf{w}) = \mathbf{w}^{T}\Phi^{T}\Phi\mathbf{w}$. Thus,
1) $\Delta f = (\mathbf{w}+\Delta \mathbf{w})^T\Phi^{T}\Phi(\mathbf{w}+\Delta \mathbf{w}) - \mathbf{w}^{T}\Phi^{T}\Phi\mathbf{w} = \mathbf{w}^T\Phi^{T}\Phi\Delta \mathbf{w} + (\Delta \mathbf{w})^T\Phi^{T}\Phi\mathbf{w} + (\Delta \mathbf{w})^T\Phi^{T}\Phi\Delta \mathbf{w};$
2)$\partial f = \mathbf{w}^T\Phi^{T}\Phi\Delta \mathbf{w} + (\Delta \mathbf{w})^T\Phi^{T}\Phi\mathbf{w};$
3)$\partial f = \mathbf{w}^T\Phi^{T}\Phi\partial \mathbf{w} + (\partial \mathbf{w})^T\Phi^{T}\Phi\mathbf{w}$.
Notice that $\mathbf{w}^T\Phi^{T}\Phi\partial \mathbf{w} = (\partial \mathbf{w})^T\Phi^{T}\Phi\mathbf{w}$. Thus, $\frac{\partial f}{\partial \mathbf{w}} = 2\mathbf{w}^T\Phi^{T}\Phi.$
Actually, your result is the gradient $\nabla_\mathbf{w} f = (\frac{\partial f}{\partial \mathbf{w}})^T$.
