I have the following analytical cost function: $$ \textit{J}(\bf{w}) = \textit{k} +\bf{q}^{T}\bf{Bw}+\bf{w}^{T}\bf{Cw}-\lambda\bf{w}^{T}\bf{Dw} $$ where $\textit{k}$ is a constant, bold lowercase variables are vectors (dimension Nx1) and bold upercase are matrices (dimension NxN). The only variable is $\bf{w}$ (others are constant). The constraint is $\bf{w}^{T}\bf{Dw}\geq0$ and $\lambda$ is the lagrange multiplier. The derivatives are:

\begin{array}{l} \frac{{\partial {J(\bf{w})}}}{{\partial {\bf{w}}}} = {{\bf{B}}^T}{\bf{q}} + \left( {{\bf{C}} + {{\bf{C}}^T}} \right){\bf{w}} - \lambda \left( {{\bf{D}} + {{\bf{D}}^T}} \right){\bf{w}} = {\bf{0}}\\ \frac{{\partial {J(\bf{w})}}}{{\partial \lambda }} = - {{{\bf{w}}}^T}{\bf{D}\bf{w}} = 0\\ \lambda \ge 0 \end{array}

Pre-multiplying the first derivative by $\textbf{w}^{T}$ results in

${{\bf{\hat w}}^T}\left[ {{{\bf{B}}^T}{\bf{q}} + \left( {{\bf{C}} + {{\bf{C}}^T}} \right){\bf{\hat w}}} \right] = 0$

As a result

${\bf{\hat w}} = {\left( { - {\bf{C}} - {{\bf{C}}^T}} \right)^{ - 1}}{{\bf{B}}^T}{\bf{q}}$

The constraint disapeared! What happened with it? Was it intrinsically satisfied? For any $\bf{D}$?

  • $\begingroup$ Why do the derivatives have to equal zero? This is not the case in a constrained optimization problem. You should have the derivative of the lagragian equal zero. Also, your notation suggests partial derivatives but the equations look like gradients. Is this a mistake or intentional? $\endgroup$ – DiegoNolan May 28 '13 at 21:23

Your last equation doesn't follow from anything you'd written above; certainly not from the equation above it, which is a scalar equation and thus can only determine at most one degree of freedom of $\hat{\mathbf w}$. If you want to get a vector equation such as the last equation, you have to solve the equation $\partial J(\mathbf w)/\partial\mathbf w=\mathbf 0$ without projecting it along a single direction, and then $\mathbf D+\mathbf D^T$ does appear in the matrix whose inverse you need to take.

  • $\begingroup$ The first equation is already the Lagrangian (cost function + constraint). Projecting along a single side direction results in the penultimate equation whose trivial solutions are /bf{w}=/bf{0} and the last one. $\endgroup$ – Marcio May 30 '13 at 10:40
  • $\begingroup$ @Marcio: I'm afraid I don't understand your comment. About the first sentence: Yes, I was aware of that. About the second sentence: What's a "side direction"? Certainly the two solutions you give are solutions of that equation, but a single scalar equation for an $N$-dimensional vector has more than two solutions. I don't understand how you're trying to infer from the mere fact that these two vectors solve that one projected equation that one of them must be the solution to your overall problem. $\endgroup$ – joriki May 30 '13 at 11:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.