Two non-co-linear vectors define a 2D subspace that passes through the origin. In 3D, you can represent a 2D subspace by its Normal. It's very easy to define the angle between the two planes: $\theta = \cos^{-1}(N_1\cdot N_2)$ (assuming unit length) and it's a simple matter to find a new plane "in between" by interpolating to find a new normal: $N_t = \frac{\sin((1-t)\theta)}{\sin(\theta)}N_1 + \frac{\sin(t\theta)}{\sin(\theta)}N_2$. Essentially, we're rotating the new Normal vector within the 2D sub-surface defined by the two original Normals.

Rotating a vector this way extends easily into higher dimensions, but a Normal defines a hyper-plane, not a linear, 2D subspace. So I can't use the same approach. If I have two pairs of orthogonal vectors defining two distinct linear 2d subspaces, is there a way to measure the angle between them or find a new subspace that is an interpolation between the two?


Working with the problem, I realized that even in 3D, interpolating the normals wouldn't be sufficient for my problem, but I would actually need to interpolate the space defined by the vector pairs. It's still straightforward to do in 3D. Referring to the image below, if I have vectors $X$ and $Y$ and wish to interpolate to $X'$ and $Y'$, it's simple if I find vectors $Z$ and $Z'$ by the cross-product approach. Then we can find $F = \left[X\ Y\ Z\right]$ and $F'=\left[X'\ Y'\ Z'\right]$. The rotation matrix making the full transform from $F$ to $F'$ would just be $F'F^T$. From that rotation matrix, it's easy to extract the axis and angle of rotation and then simply interpolate through the angle.

In higher dimensions, cross-product doesn't work. However, we can still find a point in terms of the first frame of reference and project it into the second: $P(1) = \left[X'(X\cdot P(0) + Y'(Y\cdot P(0))\right]$. Given any pair of $P(0)$ and $P(1)$, I can use spherical interpolation find a point on the shortest arc between the two points, but that's not what I need. I need to interpolate the 2D space itself and find the corresponding point within the plane. If I need to add any number of extra axes because there are multiple equally short paths, that's fine.

Points on a sphere

  • $\begingroup$ the question is not "is there a way to find an interpolation", but rather "which is a good way". If you let $e_1, f_1$ be independent vectors spanning the subspace $V_1$, and $e_2, f_2$ be independent vectors spanning $V_2$. Than for the family of subspaces $V_\lambda$ spanned by $\lambda e_1 + (1-\lambda)e_2, \lambda f_1 + (1-\lambda)f_2$ would be generically a family of subspaces interpolating between $V_1$ and $V_2$. $\endgroup$ – Willie Wong Apr 13 '11 at 17:39
  • 2
    $\begingroup$ What I suspect you want, however, is to find the one-parameter family of interpolants corresponding to a geodesic on the Grassmanian manifold. Unfortunately, a convenient representation of this family doesn't come to my mind immediately. I hope it will to someone else. $\endgroup$ – Willie Wong Apr 13 '11 at 17:41
  • $\begingroup$ Thanks, I appreciate the pointer to the Wikipedia article. I apparently need to learn a whole lot of vocabulary in order to properly understand the discussion. $\endgroup$ – JCooper Apr 14 '11 at 0:23

It takes $k$ angles to describe the relative positions of two $k$ dimensional linear subspaces. These are the principal angles. So in your case there are two angles to consider. Let $K$ and $L$ be your subspaces in $\mathbb{R}^d$, and let $P_L: \mathbb{R}^d \to L$ be the orthogonal projection onto $L$, and $P_{KL}$ its restriction to $K$. Then the singular values of $P_{KL}$ are the cosines of the principal angles.

As Willie Wong mentioned, what we are trying to do is parameterize a geodesic on a Grassmanian manifold. According to Greg Kuperberg's answer here, these principal angles give us what we want, i.e., the in-between planes are given by $(t\theta_1, t\theta_2)$ for $t \in [0,1]$.

  • 1
    $\begingroup$ I think this is a good motivation for learning about the singular value decomposition. With this picture you can see that any linear map (between inner product spaces) can be characterized, up to a scaling factor, as an orthogonal projection. between linear subspaces $\endgroup$ – yasmar Apr 13 '11 at 22:22
  • 1
    $\begingroup$ The interpolation then can be implemented in a fashion similar to Willie Wong's observation that we can interpolate between basis vectors. We want to use the singular vectors of $P_{KL}$ for these bases, and then rotate them into each other rather than just doing linear interpolation. $\endgroup$ – yasmar Apr 13 '11 at 22:29
  • $\begingroup$ Huh. I've used singular value decomposition as a tool, but I guess I don't have a good grip on what it does, since I thought it was essentially the same as principal components analysis, and it seems that's not the case. I have some learning to do before I can really understand the answers given here. $\endgroup$ – JCooper Apr 14 '11 at 0:32
  • $\begingroup$ A good reference for the SVD is Lectures 4 and 5 of Trefethen and Bau's "Numerical Linear Algebra", which you can find online here. $\endgroup$ – yasmar Apr 14 '11 at 5:20
  • $\begingroup$ If the principal angles $\theta_1$ and $\theta_2$ relate the subspaces $K$ and $L$, is there a similarly simple way to find the the angle that relates the orthonormal vectors $\vec{a}_1,\vec{a}_2$ and $\vec{b}_1,\vec{b}_2$ (as per joriki's answer). I.e., once I have rotated $K$ into $L$, how much must I rotate $\vec{a}'_1$ and $\vec{a}'_2$ (within $L$) so they become $\vec{b}_1$ and $\vec{b}_2$? $\endgroup$ – Jellby Feb 7 '16 at 11:14

I think this is the same answer as yasmar's, but from a more computational perspective.

It makes sense that the pair of vectors in the two subspaces forming the least angle with each other is distinguished and these vectors should be rotated into each other if the interpolation is to follow a geodesic. To find these vectors, you can find the stationary values of the dot product between arbitrary vectors in the subspaces,


where $\vec{a}_1$ and $\vec{a}_2$ form an orthonormal basis for the first subspace and likewise $\vec{b}_1$ and $\vec{b}_2$ for the second one. (You can easily calculate these by orthonormalization if you don't already have them.) With $g_{ij}:=\vec{a}_i\cdot\vec{b}_j$, this becomes

$$ g_{11}\cos\phi\cos\theta+ g_{12}\cos\phi\sin\theta+ g_{21}\sin\phi\cos\theta+ g_{22}\sin\phi\sin\theta\;. $$

Differentiating with respect to $\theta$ and $\phi$ yields

$$ \begin{eqnarray} -g_{11}\cos\phi\sin\theta +g_{12}\cos\phi\cos\theta -g_{21}\sin\phi\sin\theta +g_{22}\sin\phi\cos\theta &=&0\;,\\ -g_{11}\sin\phi\cos\theta -g_{12}\sin\phi\sin\theta +g_{21}\cos\phi\cos\theta +g_{22}\cos\phi\sin\theta &=&0\;, \end{eqnarray} $$

and adding and subtracting these equations leads to

$$ \begin{eqnarray} (g_{22}-g_{11})\sin(\phi+\theta)+(g_{12}+g_{21})\cos(\phi+\theta)&=&0\;,\\ (g_{22}+g_{11})\sin(\phi-\theta)+(g_{12}-g_{21})\cos(\phi-\theta)&=&0\;,\\ \end{eqnarray} $$

from which $\phi+\theta$ and $\phi-\theta$, and hence $\phi$ and $\theta$, can be determined.

That gives you the vectors $\vec{a}_<:=\cos\phi\vec{a}_1+\sin\phi\vec{a}_2$ and $\vec{b}_<:=\cos\theta\vec{b}_1+\sin\theta\vec{b}_2$ that should be rotated into each other. The vectors $\vec{a}_>$ and $\vec{b}_>$ with $\vec{a}_> \perp \vec{a}_<$ and $\vec{b}_> \perp \vec{b}_<$ also fulfill $\vec{a}_> \perp \vec{b}_<$ and $\vec{b}_> \perp \vec{a}_<$ (else the stationary point wouldn't be stationary). You can choose their signs so as to form the smaller of the two possible angles between them.

Now these pairs of vectors form two planes that are orthogonal to each other and within which the interpolation rotations should take place, with $\vec{a}_<$ being rotated into $\vec{b}_<$ and $\vec{a}_>$ being rotated into $\vec{b}_>$, both with (different) linear rates of change of the respective rotation angles.

  • $\begingroup$ What if a_1, a_2, b_1, and b_2 are all mutually orthogonal (rotating the w-x 2-plane onto the y-z 2-plane)? Doesn't that make g_{ij} always zero? That would make finding \theta and \phi a bit tricky. Did I misunderstand something? $\endgroup$ – JCooper Apr 14 '11 at 1:31
  • $\begingroup$ Is that just the special case where there are multiple possible answers and I can choose a path as when rotating one vector onto another if the vectors are co-linear? $\endgroup$ – JCooper Apr 14 '11 at 1:33
  • 1
    $\begingroup$ I think you are right, our answers are equivalent. If we think of the orthogonal projection of the unit circle in the first plane into the second plane, we get an ellipse. The vectors which define the axis of this ellipse are your $\vec{b}_{<}$ and $\vec{b}_{>}$, and they are also the left singular vectors. Likewise the vectors which project onto these are the $\vec{a}_<$ and $\vec{a}_>$, which are the right singular vectors. Also we have $a_< \cdot b_< = \cos \theta_i$, i.e. their dot product is the cosine of a principal angle, and it is a singular value. $\endgroup$ – yasmar Apr 14 '11 at 5:17
  • $\begingroup$ @JCooper If the planes are mutually orthogonal, then you are right; any path will do. In other words you don't need to first find special bases for the planes, as long as they're consistently oriented. In this case there is not a canonical way to relate vectors in one subspace to the other. $\endgroup$ – yasmar Apr 14 '11 at 5:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.