Rules of transposes for inner products

Question

Right now, I am reading a text on Nonlinear Optimization for a class, and there is a minimization example where for $A$, a matrix of dimension $m \times n$, $c \in \mathbb{R}^{n}$ and $b \in \mathbb{R}^{m}$, we have that the Lagrangian is given by $$L(x, \lambda) = \langle c, x \rangle + \langle \lambda, b-A x \rangle = \langle c - A^{T}\lambda, x \rangle + \langle b, \lambda \rangle $$

where $A^{T}$ denotes the transpose of the matrix $A$.

I know that there are inner product rules for transposes that allow us to have $ \langle c, x \rangle + \langle \lambda, b-A x \rangle = \langle c - A^{T}\lambda, x \rangle + \langle b, \lambda \rangle$, but I do not know what they are.

Could someone please explain to me why this equality is true, as well as listing the properties of inner products that I would need in order to tackle problems like this in the future (or at least some place online where I can find all of these properties)?

Thank you.

Answer 1

The rules you need to know are

$$ \langle a+b,c\rangle=\langle a,c\rangle+\langle b,c\rangle \\ \langle u,v+w\rangle=\langle u,v\rangle+\langle u,w\rangle \\\langle p,q\rangle=\langle q,p\rangle \\ \langle Ap,q\rangle=\langle p,A^Tq\rangle $$

Verify these yourself if you want. Of course because of the third rule, an alternate form of the fourth rule may be stated as $\langle p,Aq\rangle=\langle A^Tp,q\rangle$. So the derivation goes

$$ \begin{array}{l} ~\phantom{=} \langle c, x \rangle + \langle \lambda, b-A x \rangle \\ =\langle c,x\rangle+\langle \lambda,b\rangle-\langle \lambda,Ax\rangle \\ = \langle c,x\rangle-\langle \lambda,Ax\rangle+\langle \lambda,b\rangle \\ =\langle c,x\rangle-\langle A^T\lambda,x\rangle+\langle \lambda,b\rangle \\ =\langle c-A^T\lambda ,x\rangle+\langle \lambda,b\rangle \\ =\langle c-A^T\lambda,x\rangle+\langle b,\lambda\rangle \end{array} $$