Suppose that for some function $f$ we want to know the stationary points, i.e. $\frac{\partial f(\mathbf{x})}{\partial \mathbf{x}} = \mathbf{0}$.

We can define a new function $\widetilde{f}(\mathbf{x}) = \left( \frac{\partial f(\mathbf{x})}{\partial \mathbf{x}} \right)^2$, which is zero when the partial derivatives w.r.t. each $x$ is zero. For this function we have that $\frac{\partial \widetilde{f}(\mathbf{x})}{\partial \mathbf{x}} = \mathbf{2}^T \frac{\partial^2 f(\mathbf{x})}{\partial \mathbf{x}^2} \text{diag} \left( \frac{\partial f(\mathbf{x})}{\partial \mathbf{x}} \right)$

The partial derivative w.r.t. a single variable is given by $\frac{\partial \widetilde{f}(\mathbf{x}}{\partial x_i} = \frac{\partial f(\mathbf{x})}{\partial x_i} \sum\limits_j \frac{\partial^2 f(\mathbf{x})}{\partial x_i \partial x_j}$

The stationary points of $f$ are then also stationary points of $\widetilde{f}$, since when $\frac{\partial f(\mathbf{x})}{\partial \mathbf{x}} = \mathbf{0}$ it holds that $\frac{\partial \widetilde{f}(\mathbf{x})}{\partial \mathbf{x}} = \mathbf{0}$

Sometimes this may help in finding an analytic solution for $\mathbf{x}$ such that $\frac{\partial f(\mathbf{x})}{\partial \mathbf{x}} = \mathbf{0}$

Is there a name for this kind of process? Is this a generally accepted technique?

  • $\begingroup$ Is not $\frac{\partial^2f}{\partial\mathbf{x}^2}$ a matrix? How can this process "help in finding a solution"? $\endgroup$ – enzotib Jun 20 '14 at 11:55
  • $\begingroup$ Yes that is a matrix. $\frac{\partial \widetilde{f}(\mathbf{x}}{\partial x_i} = \frac{\partial f(\mathbf{x})}{\partial x_i} \sum\limits_j \frac{\partial f(\mathbf{x})}{\partial x_i \partial x_j}$. See my dit of the partial derivative of the new function. $\mathbf{2}$ is a vector of 2's. $\endgroup$ – Angelorf Jun 20 '14 at 13:12
  • $\begingroup$ Actually $\widetilde{f}$ has a minimum where $f$ has any stationary point. As such we can use numerical optimization techniques for $\widetilde{f}$ which find minima and don't easily find saddle points, as some numerical optimization techniques do. $\endgroup$ – Angelorf Jun 20 '14 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.