Right now, I am reading a text on Nonlinear Optimization for a class, and there is a minimization example where for $A$, a matrix of dimension $m \times n$, $c \in \mathbb{R}^{n}$ and $b \in \mathbb{R}^{m}$, we have that the Lagrangian is given by $$L(x, \lambda) = \langle c, x \rangle + \langle \lambda, b-A x \rangle = \langle c - A^{T}\lambda, x \rangle + \langle b, \lambda \rangle $$

where $A^{T}$ denotes the transpose of the matrix $A$.

I know that there are inner product rules for transposes that allow us to have $ \langle c, x \rangle + \langle \lambda, b-A x \rangle = \langle c - A^{T}\lambda, x \rangle + \langle b, \lambda \rangle$, but I do not know what they are.

Could someone please explain to me why this equality is true, as well as listing the properties of inner products that I would need in order to tackle problems like this in the future (or at least some place online where I can find all of these properties)?

Thank you.


The rules you need to know are

$$ \langle a+b,c\rangle=\langle a,c\rangle+\langle b,c\rangle \\ \langle u,v+w\rangle=\langle u,v\rangle+\langle u,w\rangle \\\langle p,q\rangle=\langle q,p\rangle \\ \langle Ap,q\rangle=\langle p,A^Tq\rangle $$

Verify these yourself if you want. Of course because of the third rule, an alternate form of the fourth rule may be stated as $\langle p,Aq\rangle=\langle A^Tp,q\rangle$. So the derivation goes

$$ \begin{array}{l} ~\phantom{=} \langle c, x \rangle + \langle \lambda, b-A x \rangle \\ =\langle c,x\rangle+\langle \lambda,b\rangle-\langle \lambda,Ax\rangle \\ = \langle c,x\rangle-\langle \lambda,Ax\rangle+\langle \lambda,b\rangle \\ =\langle c,x\rangle-\langle A^T\lambda,x\rangle+\langle \lambda,b\rangle \\ =\langle c-A^T\lambda ,x\rangle+\langle \lambda,b\rangle \\ =\langle c-A^T\lambda,x\rangle+\langle b,\lambda\rangle \end{array} $$

  • $\begingroup$ thank you! This was extremely helpful! Also, if $k \in \mathbb{R}$, we have $k \langle p, q \rangle = \langle kp, q \rangle$, and then because of the fourth rule this is also $= \langle p, kq \rangle$, right? $\endgroup$ – user100463 Dec 4 '17 at 14:22
  • $\begingroup$ I also wanted to ask you how to multiply out $\langle c-A^{T}\lambda, x \rangle$. $\endgroup$ – user100463 Dec 4 '17 at 15:51
  • $\begingroup$ First comment: yes. Second comment: what do you mean? $\endgroup$ – anon Dec 4 '17 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy