Computing a quadratic form For context I am taking an optimization class and I am unfamiliar to most basic concepts of vector calculus hence my question which will seem trivial to most.
I have trouble understanding the following equality :
\begin{align}
\frac{1}{2}(x+\alpha d)^\top Q (x+\alpha d) + q^\top (x+\alpha d) - \frac{1}{2}x^\top Q x - q^\top x = \frac{1}{2}\alpha^2d^\top Q d + \alpha (Q x + q)^\top d
\end{align}
I have trouble understanding where the term $\alpha(Qx)^\top d$ comes from. I know from elementary algebra that $(a+b)^2 = a^2 + b^2 + 2ab$. Does that mean that the following equality holds for vector calculus ?
\begin{align}
(a+b)^\top Q (a+b) = a^\top Q a + b^\top Q b + 2a^\top Q b \quad a,b \in \mathbb{R}^n \quad, Q\in \mathbb{R}^{n\times n}
\end{align}
 A: In general for arbitrary $a,b\in\mathbb{R}^n$ and $Q\in\mathbb{R}^{n\times n}$ you have the following (applying distributivity):
$$
(a+b)^{\top}Q(a+b)=
(a^{\top}+b^{\top})Q(a+b)=
(a^{\top}+b^{\top})(Qa+Qb)=
a^{\top}Qa+b^{\top}Qa+a^{\top}Qb+b^{\top}Qb.
$$
In general products with matrices do not commute, i.e. $b^{\top}Qa\neq a^{\top}Qb$ and $b^{\top}Qa+a^{\top}Qb\neq 2a^{\top}Qb$.
However it's good that you mentioned your context, optimization. In optimization $Q$ is usually symmetric, i.e. $Q^\top=Q$. Using this special condition we can achieve equality. Note that $b^{\top}Qa\in\mathbb{R}^{1\times 1}=\mathbb{R}$ which can be considered as a number. Transposing a number will not change it. Therefore (note that transposition reverses multiplication order and double transposition cancels):
$$
b^{\top}Qa=
(b^{\top}Qa)^\top=
a^\top Q^\top (b^\top)^\top=
a^\top Q^\top b\stackrel{\text{assumption }Q^\top=Q}{=}
a^{\top}Qb.
$$
If your $Q$ is symmetric, which seems highly plausible from the context, it works out. If not, then your equality would be wrong in general.
