# An optimization problem involving orthogonal matrices

Let $X\in\mathbb{R}^{3\times 3}$ be an orthogonal matrix. Then $\mathrm{vec}X\in\mathbb{R}^9$ is a 9 by 1 vector formed by stacking the columns of the matrix $X$ on top of one another. Given a matrix $A\in\mathbb{R}^{9\times 9}$, find the optimal orthogonal matrix $X$ minimizing the following objective function. $$J=\left(\mathrm{vec}X\right)^T A \mathrm{vec}X$$ I think Kronecker product may be useful for solving this problem. Does a closed-form solution exist? If not, is it possible to solve it iteratively? Thanks.

EDIT

a) Here $A$ doesn't have any special property. But it is also acceptable if solutions can be obtained by adding some properties on $A$.

b) In the original problem, $X$ is constrained as a rotation matrix. But I think that would be even harder, so I put $X$ as an orthogonal matrix herein. Of course, optimal rotation matrices are better.

-
Does $A$ has any property, e.g. being positive definite? – Hauke Strasdat Oct 9 '11 at 15:28
@Hauke Strasdat: Thanks, I've edited the question about this point. – Shiyu Oct 9 '11 at 15:36
Note sure whether there is a closed form solution. But you could solve it iteratively (e.g. using the Newton method or Gradient Descent) and the Lie algebra so3. Is X a proper rotation $\det(X)=1$ or can it also be a reflection? – Hauke Strasdat Oct 9 '11 at 15:42
You can preserve orthogonality, if you do the update in the tangent space (Lie algerba). Thus, you calculate the gradient: $\nabla J =\frac{\partial }{\partial \epsilon} \text{vec} (\exp(\epsilon)X)^\top A \text{vec} (\exp(\epsilon)X)|_{\epsilon=0}$ with $\exp(\cdot)$ being the Rodriguez formula. Now you can iteratively update $X$ using gradient descent: $X' = \exp(-\alpha\nabla J) X$. – Hauke Strasdat Oct 9 '11 at 16:23
On the issue of rotations vs. reflections, note since X is of odd dimension and can be replaced wlog in the objective function by -X if necessary, there is no problem restricting attention to det(X)=1. – hardmath Oct 9 '11 at 16:36