# Vector , Matrix multiplication properties

I am quite weak when it comes to matrices and would like some help. I am trying to re-derive the weighted least squares below. I am not sure how to go from (2) to (3). How can you just shift the variables around ? I tried transposing the equation but I still get something very different. Also from (4) to (5), how does $wxx^T$ become $X^TWX$ ?

How does the magic happen ? Is there a reference book with all the magical matrix tricks ?

To go from (2) to (3), first note that the rightmost summation is a linear combination of vectors $\tilde{\mathbf{x}}_n$, with coefficients with happen to be scalar products of the form $\beta^T \tilde{\mathbf{x}}_n$. But, this scalar product can also be written as $\tilde{\mathbf{x}}_n^T \beta$, so the terms become: $w_n (\tilde{\mathbf{x}}_n^T \beta) \tilde{\mathbf{x}}_n$. You can easily check that this really equals $w_n \tilde{\mathbf{x}}_n \tilde{\mathbf{x}}_n^T \beta$, which appears in (3).
Now, the passage from (4) to (5) suggests that $\mathbf{W}$ is a diagonal matrix containing the numbers $w_n$ on its diagonal and also that the vectors $\tilde{\mathbf{x}}_n$ are the columns of $\tilde{\mathbf{X}}^T$. Hence, $\tilde{\mathbf{X}}^T \mathbf{W}$ yields the linear combination $\sum_n w_n \tilde{\mathbf{x}}_n$. If you multiply $\tilde{\mathbf{X}}^T \mathbf{W}$ from the right by $\tilde{\mathbf{X}}$, what you get is a linear combination of outer products of columns of $\tilde{\mathbf{X}}^T$ by rows of $\tilde{\mathbf{X}}$, which are the vectors $\tilde{\mathbf{x}}_n^T$. The coefficients of this linear combination are precisely the numbers $w_n$. Hence, (4) and (5) are really equal.