Minimization of $\text{tr} (W^TMW)-\text{tr}(NW)$ subject to $W^TW=I$ Is there a closed-form solution for finding W that minimizes the objective function: $\text{tr} (W^TMW)-\text{tr}(NW)$ subject to $W^TW=I$ where $M$ and $N$ are fixed matrices. I find it difficult to solve using Lagrange.
 A: Assuming that $M$ is symmetric positive definite, then we can use the vec operator and the Kronecker product to translate the problem to a linear least squares problem over a sphere.
Let $\mathbf{w}=\mathrm{vec}(W)$ and $\mathbf{q}=\frac{1}{2}\mathbf{M}^{-1}\mathrm{vec}(N^T)$, where $\mathbf{M}=I\otimes M$ (the block diagonal matrix with $M$ repeated $k$-times on the diagonal with $k$ being the number of rows of $N$ and columns of $W$). Then
$$
\mathrm{tr}(W^TMW) = \mathbf{w}^T\mathbf{M}\mathbf{w},
\qquad
\mathrm{tr}(NW) = 2\mathbf{q}^T\mathbf{M}\mathbf{w}.
$$
Note that
$$
\|\mathbf{w}-\mathbf{q}\|_{\mathbf{M}}=(\mathbf{w}-\mathbf{q})^T\mathbf{M}(\mathbf{w}-\mathbf{q})
=\mathrm{tr}(W^TMW)-\mathrm{tr}(NW)+\mathbf{q}^T\mathbf{M}\mathbf{q},
$$
so minimizing $\|\mathbf{w}-\mathbf{q}\|_{\mathbf{M}}$ is equivalent to minimizing $\mathrm{tr}(W^TMW)-\mathrm{tr}(NW)$ since $\mathbf{q}^T\mathbf{M}\mathbf{q}$ is independent of $W$.
The condition $W^TW=1$  translates simply to $\|\mathbf{w}\|_2=1$.
So the problem is equivalent to: find $\mathbf{w}$ such that
$$
\|\mathbf{w}-\mathbf{q}\|_{\mathbf{M}}=\min_{\tilde{\mathbf{w}}}\|\tilde{\mathbf{w}}-\mathbf{q}\|_{\mathbf{M}}\quad\text{subject to}\quad\|\mathbf{w}\|_2=1.
$$
This is essentially a sphere constrained least squares problem. You can find a solution method, e.g., in Matrix Computations by Golub and Van Loan.
