I am reading the paper "Wasserstein Geometry of Gaussian measures" by Asuka Takatsu (section 3 is of interest to me) and I have difficulties understanding how the metric is used.

In particular, I am wondering the following :

  • if I take the square root of the covariance matrices of my gaussians, does the space become Euclidean ? I feel this should be the case as the metric $tr(XY)$ for $X$ and $Y$ two symmetric matrices allow to recover than $\frac{d}{dt}\langle\dot\gamma,\dot\gamma\rangle=0$ if $\gamma^2$ is a geodesic in the space of covariance matrices (if $\gamma^2(t)$ is a geodesic, it can be written $\gamma(t)=A+tT$, so differentiating by $T$ gives the tangent vector which is independant of the time).
    However, if this space was Euclidean, wouldn't it mean that an interpolation between a covariance matrix $V$ and another covariance matrix $U$ necessarily be given by $C(t) = (V^{\frac{1}{2}} + t (U^{\frac{1}{2}}-V^{\frac{1}{2}}))^2$ ? This is not the case as geodesics are given by $C(t) = (V^{\frac{1}{2}}+t(U^{\frac{1}{2}}(U^{\frac{1}{2}}VU^{\frac{1}{2}})^{-\frac{1}{2}}U^{\frac{1}{2}}V^{\frac{1}{2}} - V^{\frac{1}{2}}))^2$ (except in 1D where these are equivalent)
  • In which space is the metric $g_V(X,Y)=tr(XVY)$ used ? I thought it was directly in the space of covariance matrices but this doesn't seem to be the case. For instance $\frac{d}{dt}g(\dot\gamma,\dot\gamma)\neq0$ : if $\gamma$ is a geodesic in the space of covariance matrices, then $\gamma(t) = (A+tT)^2$ and then $\dot\gamma = TA+AT+2tTT$ and $\frac{d}{dt}g(\dot\gamma,\dot\gamma)\neq 0$



Let $\mathcal N_0^d$ be the space of positive definite matrices of size $d$. Also let $\mathcal{G}=GL(d,\mathbb R)$ as in the paper. We have the natural surjection $\Pi: \mathcal{G}\to \mathcal N_0^d$ defined by $\Pi(A)=A^TA$. The general linear group $\mathcal{G}$ has a Riemannian metric $G$ as an open submanifold of $\mathbb R^{d\times d}$; that is, the tangent space at every point of $\mathcal{G}$ is identified with $\mathbb R^{d\times d}$ which carries its standard metric. The author wants to "push forward" the metric $G$ by $\Pi$, which means putting a metric $g$ on $\mathcal N_0^d$ such that $\Pi$ becomes a Riemannian submersion. Of course, metrics do not normally push forward. The authors pretends that this issue does not exist and writes

We define a Riemannian metric $g$ on $\mathcal N_0^d$ by $$g_{\Pi}(d\Pi(Z),d\Pi(W))=G(Z_{\mathcal H},W_{\mathcal H}).$$

We are really defining the metric at $\Pi(A)$ for some $A\in \mathcal{G}$, and since $\Pi$ is not a bijection, one should explain why the choice of $A$ does not matter. Instead, $A$ is simply omitted from the formula.

The reason why $\Pi$ is able to push $G$ forward can be stated as: (1) the orthohonal group $O(d,\mathbb R)$ acts on $\mathcal{G}$ by left multiplication; (2) this action is isometric; (3) the fibers of $\Pi$ are orbits under this action.

On the next page there is an explicit formula (3.4) for $g$. Maybe it would be more logical to introduce $g$ by (3.4) and then check that it makes $\Pi$ a Riemannian submersion. Whatever. Regarding (3.4) it is important to emphasize (and the author does it) that the tangent vectors $X$ and $Y$ in (3.4) are not written in coordinates of the ambient space $\mathbb R^{d\times d}$. This is why your computation in the 2nd bullet item did not give the right result.

Let's analyze the case $d=1$. Here $\mathcal N_0^1=(0,\infty)$, covered by the map $x\to x^2$ which pushes forward the Euclidean metric on $\mathcal G=\mathbb R\setminus \{0\}$. The result is $\mathcal N_0^1 = ((0,\infty), \frac12 x^{-1/2}dx)$. Applied carelessly, the formula (3.4) would give a wrong result $g(dx,dx)=x\,dx^2$ instead of the correct one $g(dx,dx)=\frac{1}{4x}dx^2$. One has to transform the tangent vectors (multiplying them by $A^{-1}$ or something) before sticking them into $\operatorname{Tr}(XVY)$.

You also mentioned the square root map $r: \mathcal N_0^d\to \mathcal G$. This is a right inverse of $\Pi$ in the sense that $\Pi\circ r=\operatorname{id}_{\mathcal N_0^d}$. However, $r$ is not an isometric embedding. The reason is that the range of $r$ (which is the set of all positive definite matrices) is not horizontal with respect to $\Pi$: its tangent space is not the subspace on which $d\Pi$ is isometric. Here is a simple illustration in two dimensions: $(x,y)\mapsto x$ is a Riemannian submersion from $\mathbb R^2$ onto $\mathbb R$, the map $x\mapsto (x,e^x)$ is its right inverse, but it is not an isometric embedding of $\mathbb R$ into $\mathbb R^2$.

Here is an explicit calculation that confirms the statement in the preceding paragraph. Let $A=\begin{pmatrix}2 & 1 \\ 1 & 2 \end{pmatrix}$. The tangent space of $\mathcal{G}$ at $A$ is decomposed into the kernel of $d\Pi$, which is the span of $B=\begin{pmatrix}1 & 2 \\ -2 & -1 \end{pmatrix}$, and its orthogonal complement, on which $d\Pi$ is isometric. Since $B$ is not skew-symmetric, its orthogonal complement does not coincide with the space of symmetric matrices, i.e., with the tangent space to the range of $r$.

[Added] It's tempting to think that the metric on $\mathcal{N}_0^d$ should be $g_V(X,Y)=\frac{1}{4}\operatorname{Tr}(XV^{-1}Y)$, written without any coordinate tricks: $X$ and $Y$ are tangent vectors to $\mathcal{N}_0^d$ at $V$, which means they are symmetric matrices. But it does not seem to work out this way...

I'll try follow the idea of computations in the paper without necessarily agreeing with its details. Fix $V\in \mathcal{N}_0^d$ and let $A=V^{1/2}\in \mathcal{G}$. Given two vectors $X$ and $Y$ in $T_V \mathcal{G}$ (my notation for tangent space), we want to pull them back to horizontal vectors in $T_A \mathcal{G}$, and take the inner product there.

  1. What are the vertical vectors $Z$ in $T_A \mathcal{G}$? They satisfy $(A+Z)^T (A+Z)=A^TA$ up to second order, which means $AZ+Z^TA=0$. Rewriting this as $AZ+(AZ)^T=0$, we see that $AZ$ is skew-symmetric.
  2. What are the horizontal vectors $W$ in $T_A \mathcal{G}$? They are orthogonal to vertical vectors: $\operatorname{Tr}(Z^TW)=0$. Writing the latter as $\operatorname{Tr}((AZ)^T (A^{-1}W))=0$, we conclude that $A^{-1}W$ must be symmetric. Let's record it by saying that $W=A\widehat W$ where $\widehat W$ is symmetric.
  3. The image of horizontal vector $A=A\widehat W$ under $d\Pi_A$ is $AW+W^TA=A^2\widehat{W}+\widehat{W}A^2 = V\widehat{W}+\widehat{W}V$. This is a symmetric matrix, of course.
  4. Given a symmetric matrix $X$, we want to solve $V\widehat{W}+\widehat{W}V=X$ for unknown symmetric matrix $\widehat{W}$. Unable to so do explicitly (?), we resign to denoting the solution as $\widehat{W}_X$. We also have $\widehat{W}_Y$ for our other tangent vector.
  5. So, $g_V(X,Y) = \operatorname{Tr} ((A\widehat{W}_X)^T(A\widehat{W}_Y)) = \operatorname{Tr}(\widehat{W}_X V \widehat{W}_Y)$.

The result matches the paper, except that the "coordinate transformation" was spelled out. I wonder if $\widehat{W}_X$ can be reasonably written in terms of $X$...

| cite | improve this answer | |
  • $\begingroup$ Thank you very much for this detailed answer. I am not sure how you obtained in the 1D example that $g(dx,dx)=\frac{1}{4x}dx^2$. I would have thought that $g(dx,dx)=tr((\frac{1}{2}x^{-\frac{1}{2}}dx).x.(\frac{1}{2}x^{-\frac{1}{2}}dx)) = \frac{1}{4}dx^2$ (otherwise, where did the $V$ in $tr(XVY)$ go?). If I transform the tangents vectors by right-multiplying by $A^{-1} = V^{-\frac{1}{2}}$, I obtain the metric $g(X,Y)=tr(XV^{-\frac{1}{2}}VV^{-\frac{1}{2}} Y) = tr(XY)$ which is euclidean and would mean that this space is euclidean, right?... I feel lost :'( Thanks again! $\endgroup$ – WhitAngl Dec 25 '12 at 6:53
  • $\begingroup$ @WhitAngl In the 1D case the map $\Pi$ has local inverse (square root), so I pulled back the flat metric on $\mathbb R\setminus \{0\}$ by the square root... You can see that the result is correct by checking that $\gamma(t)=t^2$ is a geodesic with respect to this metric. I did not succeed in reconciling this result with the author's formula tr(XVY), even in one dimension... that's why there is "or something" in my answer. $\endgroup$ – user53153 Dec 25 '12 at 7:00
  • $\begingroup$ Oh ok I understand the pullback from euclidean to your 1D metric then (I already checked your calculation was correct on the geodesic ;) ). But how would that apply in higher dimension to get a metric using coordinates in the ambient space? I don't manage to extend your reasonning for, say, 2x2 matrices ($d\Pi_A(X)=A^TX+X^TA$, which has no inverse, which is why I think she introduces the horizontal subspace). If you prefer, we can briefly chat! thanks! $\endgroup$ – WhitAngl Dec 25 '12 at 7:15
  • $\begingroup$ @WhitAngl Of course we don't have inverse when $d>1$. I expanded computations in my answer (see [added]). They generally agree with what the paper does, except it again sweeps subtle things under the rug, as if hoping the reader will not notice. $\endgroup$ – user53153 Dec 25 '12 at 8:09
  • $\begingroup$ hehe - I am the kind of reader that wouldn't notice ;) Thanks again for the answer! The addendum seems to be very clear (I'll need to read it again with more sleep, but this seems to be both clear and answer my questions). Thanks! $\endgroup$ – WhitAngl Dec 25 '12 at 8:19

when V is diagonal, the equation $V\hat{W}_X+\hat{W}_XV=X$ can be solved on the canonical basis of symmetric matrices. As the Wasserstein metric is invariante by $RVR^t$, we obtain the metric for any matrix V.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.