I am trying to solve the following optimization problem in $W \in \mathbb{R}^{D \times d}$

$$\begin{array}{ll} \text{maximize} & \displaystyle\sum_{k=1}^K\text{Tr}(W^\top A_k WB_k)\\ \text{subject to} & W^\top W = I_d\end{array}$$

where $A_k \in \mathbb{S}^{D\times D}$ and $B_k \in \mathbb{S}^{d\times d}$ are given symmetric and positive definite (SPD) matrices.

I think it can be solved by Riemannian optimization methods (more specifically, Stiefel manifold). But I am wondering if there is an analytical solution, or using Lagrange multiplier (i.e., add a Lagrange term $\text{Tr}(S(W^\top W-I_d))$, where $S$ is a symmetric matrix as the Lagrange multipliers). When $K=1$, there is an analytical solution using eigendecomposition of $A_1$ and $B_1$. How about $K>2$?

Thank you!

  • $\begingroup$ Is $W$ thin or fat? $\endgroup$ – Rodrigo de Azevedo Feb 10 '18 at 9:18
  • $\begingroup$ @RodrigodeAzevedo No. $W$ is a tall matrix, so that we have $W^\top W = I_d$ $\endgroup$ – user3138073 Feb 10 '18 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.