# Nonlinear optimization with rotation matrix constraint

I'm trying to optimize the equation

|| R - W || = minimum


where W is a predetermined 3x3 matrix and R is the 3x3 matrix that I'm trying to optimize, with the constraint that it must be a rotation matrix. My knowledge of linear algebra is pretty limited, so I'm not even sure where to begin on approaching this problem?

-
Type "optimization on manifolds" into google and you will get a lot of hits with ideas you can try. The set of rotation matrices ("special orthogonal" matrices) form a group, in your case $SO(3)$, which is a Lie group. If you try newton methods, following some curve on the manifold, the derivative along the curve will always be a skew-symmetric matrix ("The Lie Algebra of the special orthogonal group is the algebra of skew-symmetric matrices"). So a Newton method should have skew-symmetric updates. –  kjetil b halvorsen Oct 5 '12 at 0:45

The answer is $R=UV',$ where $W=USV'$ is the singular value decomposition of $W.$
Assume the norm in $||R-W||^2$ is the Frobenius norm, so $||R-W||^2=\mbox{Trace}((R-W)'(R-W))= \mbox{Trace}(R'R-R'W-W'R+W'W).$ Since $R'R=I$ and $W'W$ is constant, the problem is reduced to maximizing $\mbox{Trace}(WR')= \mbox{Trace}(USV'R')$, where $W=USV'$ is the SVD of $W,$ so $S$ is non-negative and diagonal while $U$ and $V$ are orthogonal matrices. Letting $Q=V'R'U,$ and using the trace property $\mbox{Trace}(AB)=\mbox{Trace}(BA),$ the last expression is $\mbox{Trace}(SQ),$ and we want to maximize this over all rotations $Q.$ Multiplying $S$ by $Q$ rotates the rows of the diagonal matrix $S,$ and thus preserves their length. The length of each row of $S$ is the corresponding diagonal element of $S,$ thus any $Q$ other than $I$ can only reduce the trace of $SQ.$ Thus $Q=I,$ so $V'R'U=I$, so $R=UV'.$
Welcome to Math.SE! Thank you for your contribution. This site supports basic TeX, so you can type, e.g., $W=USV'$. Also, if you find time to edit your answer to include more reasoning, the OP will probably learn more. –  user53153 Dec 30 '12 at 21:56