Linear algebra: Minimizing a quadratic form under linear constraints Minimizing a quadratic form under linear constraints
Let $\textbf{A}\in \mathbb{R}^{n\times n}$ be symmetrical positive definite. Let us minimize
$$
f(\textbf{x}) := 2^{-1}\textbf{x}^\top\textbf{A}\textbf{x}
$$
over all $x \in \mathbb{R}$ fulfilling the condition $\textbf{Bx} = \textbf{c}$, where $\textbf{B}$ is a matrix in $\mathbb{R}^{q\times n}$ with rank $q\leq n$, and $\textbf{c}$ is a vector in $\mathbb{R}^q$. For $\pmb{\lambda} \in \mathbb{R}^q$, let us minimize the function $L(\textbf{x}, \pmb{\lambda}) = f(\textbf{x}) + \pmb{\lambda}^\top \textbf{Bx}$ expanded below:
\begin{align*}
L(\textbf{x}, \pmb{\lambda}) &= 2^{-1}\textbf{x}^{\top}\textbf{Ax} + (\textbf{B}^\top \pmb{\lambda})^\top\textbf{x} \quad \star 1\\
&= 2^{-1}\big(\textbf{x}^{\top}\textbf{Ax} + 2(\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}) \textbf{A}\textbf{x}\big) \quad \star 2\\
&= 2^{-1}(\textbf{x}+ \textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda})^\top\textbf{A}(\textbf{x}+ \textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}) - 2^{-1}\pmb{\lambda}^\top\textbf{B}\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda} \quad \star 3
\end{align*}
$L(\cdot, \pmb{\lambda})$ is minimized exactly when $\textbf{x}$ equals $\textbf{x}_{\lambda} := -\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}$. Furthermore,
$$
\textbf{Bx}_\lambda= -\textbf{BA}^{-1}\textbf{B}^\top\pmb{\lambda}
$$
and equals $\textbf{c}$ if and only if $\pmb{\lambda} = -(\textbf{B}\textbf{A}^{-1}\textbf{B}^\top)^{-1}\textbf{c}$. As a consequence, the problem of origin has the unique solution
$$
\textbf{x}^\star := \textbf{A}^{-1}\textbf{B}^\top(\textbf{B}\textbf{A}^{-1}\textbf{B}^\top)^{-1}\textbf{c} \quad \text{with} \quad f(x^\star) = 2^{-1}\textbf{c}(\textbf{B}\textbf{A}^{-1}\textbf{B}^\top)^{-1}\textbf{c} 
$$
Here is what I don't understand:


*

*How can you go from $\star 1$ to $\star 2$?

*How can you go from $\star 2$ to $\star 3$?

 A: There's a typo in $\star2$.  It should read
$$
2^{-1}\big(\textbf{x}^{\top}\textbf{Ax} + 2(\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda})^\top \textbf{A}\textbf{x}\big)\ .
$$
But, in any case, if you want to show that $\ \star1\ $ and $\ \star3\ $ are equal, getting there by way of $\ \star2\ $ (even as amended) seems to me to be an unnecessarily roundabout way of going about it. A much simpler way of doing it is just to expand out the expression for $\ \star3\ $ and appeal to some well-known matrix identities.
Using the fact that $\ \left(\textbf{C}\textbf{D}\right)^\top\ = \textbf{D}^\top\textbf{C}^\top\ $ and $\ \left(\textbf{C}^\top\right)^\top = \textbf{C}\ $for any matrices $\ \textbf{C}\ $ and  $\ \textbf{D}\ $ for which the product is well-defined, we get
\begin{eqnarray}
\left(\textbf{B}^\top\pmb{\lambda}\right)^\top &=& \pmb{\lambda}^\top\textbf{B}\mbox{ , and}\\
\left(\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}\right)^\top&=& \pmb{\lambda}^\top\textbf{B}\left(\textbf{A}^{-1}\right)^\top\\
&=& \pmb{\lambda}^\top\textbf{B}\textbf{A}^{-1}\ ,
\end{eqnarray}
because the symmetry of $\ \textbf{A}\ $ implies that $\ \textbf{A}^{-1}\ $ is also symmetric.  And because $\ \pmb{\lambda}^\top\textbf{B}\textbf{x}\ $ is a scalar, we have
\begin{eqnarray}
\pmb{\lambda}^\top\textbf{B}\textbf{x} &=& \left(\pmb{\lambda}^\top\textbf{B}\textbf{x}\right)^\top\\
&=& \textbf{x}^\top\textbf{B}^\top\pmb{\lambda}
\end{eqnarray}
So,
\begin{eqnarray}
 \star 3 \quad &=&2^{-1}(\textbf{x}+ \textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda})^\top\textbf{A}(\textbf{x}+ \textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}) - 2^{-1}\pmb{\lambda}^\top\textbf{B}\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}\\
&=& 2^{-1}\big(\textbf{x}^{\top}\textbf{Ax} + \pmb{\lambda}^\top\textbf{B}\textbf{A}^{-1} \textbf{A}\textbf{x} + \textbf{x}^{\top}\textbf{A}\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}\\
& &\ \ \ \ \ \ \ \ \ \ +\pmb{\lambda}^\top\textbf{B}\textbf{A}^{-1}\textbf{A}\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}\big) - 2^{-1}\pmb{\lambda}^\top\textbf{B}\textbf{A}^{-1}\textbf{B}^\top\pmb{\lambda}\\
 &=& 2^{-1}\big(\textbf{x}^{\top}\textbf{Ax} + \pmb{\lambda}^\top\textbf{B}\,\textbf{x} + \textbf{x}^\top\textbf{B}^\top\pmb{\lambda}\big)\\
&=& 2^{-1}\textbf{x}^{\top}\textbf{Ax} + \left(\textbf{B}^\top\pmb{\lambda}\right)^\top\textbf{x}\quad\star 1\ ,
\end{eqnarray}
by appealing to the identities listed above.
