What is the essential difference between classical and quantum information geometry? This question may be a little subjective, but I would like to understand, from a geometric perspective, how the structure of quantum theory differs from that of classical probability theory.
I have a good understanding of classical information geometry, and a reasonable understanding of quantum theory, for finite Hilbert spaces at least. (I am mostly only interested in finite spaces for now.)
I understand information geometry mostly from the perspective of Amari, who essentially derives everything from the "Pythagorean theorem". This says that we can treat the Kullback-Leibler divergence as a kind of squared distance. That is, if we have classical probability distributions $P$, $Q$ and $R$ such that
$$
D_{KL}(R\|Q) + D_{KL}(Q\|P) = D_{KL}(R\|P)
$$
then we may interpret $P$, $Q$ and $R$ as points that lie on the corners of a right triangle, with $Q$ at the right angle. I like to think of this as a definition of orthogonality rather than a theorem, and much of the rest of information geometry (mixture families and exponential families, the Fisher metric, dually flat spaces, and so on) can be seen as following from it.
Unfortunately Amari's book doesn't cover quantum information geometry. Here we have the quantum relative entropy, $\mathrm{Tr}\,\rho(\log \rho - \log \sigma)$, which generalises the Kullback-Leibler divergence. Presumably a similar Pythagorean interpretation can be made, with presumably gives rise to a similar dually flat structure, with quantum analogs of the Fisher metric, mixture and exponential families, and so on.
Here is my question: looking at these two objects purely as geometric spaces, is it possible to point at one or a few key defining differences between them?
Here is one observation as a possible starting point. In the quantum case, if we fix a basis and only consider distributions that are mixtures of the eigenstates, then we have a manifold that is exactly equivalent to the classical probability simplex. However, this works for any unitary basis, and so the quantum case has a kind of rotational symmetry that's lacking in the classical case.
I suspect this is not the only difference, however, and so I am looking for a characterisation of the differences between the classical probability simplex and its quantum analog, in terms of their properties as geometric manifolds.
(I would also appreciate pointers to good introductions to quantum information geometry, ideally pitched at a similar level to Amari's book and available online.)
 A: In this answer, I will focus on finite-dimensional systems since the infinite-dimensional case is very difficult to handle because of technicalities. Moreover, at the moment, I would say there is no fully satisfactory infinite-dimensional Information Geometry, neither classical nor quantum. Of course there are beautiful and deep results here too, but they are very few when compared to the finite-dimensional ones, and, more importantly, the overall picture does not seem to me to be harmonious (this is my personal opinion based mainly on my ignorance, and I would really appreciate any type of suggestion on this topic).
From a purely geometrical point of view, the space of probability distributions on a finite sample space $\mathcal{X}$ can be idenitified with the unit simplex in $\mathbb{R}^{n}$ where $n$ is the cardinality of $\mathcal{X}$. This set is not a smooth manifold, but is a smooth manifold with corners.
On the other hand, in the case of finite-level systems, the quantum counterpart of the simplex is the space of density operators on the Hilbert space $\mathcal{H}$ of the system, that is, positive semidefinite linear operators with unit trace. Density operators are also referred to as quantum states. If $\mathrm{dim}(\mathcal{H})=2$, then the set of quantum states is a closed 3-dim ball, hence, a manifold with boundary. However, when $\mathrm{dim}(\mathcal{H})>2$, then the set of quantum states is a stratified manifold (see here), which is something more complex than a manifold with corner.
From the point of view of Information Geometry, since we want to use tools from "standard differential geometry", we are not interested in the whole simplex nor in the whole space of quantum states, and we focus on their respective maximal submanifold, specifically, the manifold of strictly positive probability vectors $\Delta_{+}$ for the simplex, and the space of strictly positive (invertible) density operators $\mathcal{S}_{+}$ for the space of quantum states. These objects are smooth manifolds in the "standard" sense, and it is here that Information Geometry in the sense of Amari takes place (by the way, in the book by Amari and Nagaoka, there is a chapter devoted to the quantum case).
The first thing we have to note is that the Riemannian aspects of the Information Geometry of $\Delta_{+}$ and $\mathcal{S}_{+}$ are very different. Indeed, the Riemannian metric tensor we should consider on $\Delta_{+}$ is the Fisher-Rao metric tensor, and Cencov, in an exquisite book, proved it is unique (up to an overall multiplicative constant). This uniqueness is defined with respect to its the behaviour under Markov maps (coarse grainings). On the other hand, if we try to prove a quantum counterpart of Cencov theorem in which we replace $\Delta_{+}$ with $\mathcal{S}_{+}$ and Markov maps with Completely-Positive and trace-preserving maps, we obtain a striking result: there is an infinite number of inequivalent Riemannian metric tensors on $\mathcal{S}_{+}$. This beautiful result has been proved by Petz here.
This is already a big difference between classical and quantum information geometry, but we are not done yet. Indeed, it is well-known that the Fisher-Rao metric tensor is a "sort of second order expansion" of the Kullback-Leibler relative entropy. Roughly speaking, writing $D_{KL}(\mathbf{p},\mathbf{q})$ for the Kullback-Leibler divergence and $g_{jk}$ for the j-th and k-th component of the Fisher-Rao metric tensor, we have
$$
g_{jk}\,:=\,-\left(\frac{\partial^{2}}{\partial p^{j}\partial q^{k}}\,D_{KL}\right)_{\mathbf{p}=\mathbf{q}}\,=\,\delta_{jk}\frac{1}{p^{j}}.
$$
The same is true if we replace the Kullback-Leibler divergence with any f-divergence.
However, the same is not true in the quantum case where the counterpart of the f-divergences are the so-called relative g-entropies introduced here, and it turns out that the "second order expansion" depends on the specific relative g-entropy we consider, and, as g varies, we recover all Riemannian metric tensors classified by Petz. For instance, the Bures distance leads to the Bures-Helstrom-Uhlmann metric tensor (something similar was independently found by Cantoni) which is of capital importance in quantum parameter estimation; the Wigner-Yanase skew information leads to a metric which is the pullback with respect to the square root map on positive operators of the round metric on a suitably big sphere (the same instance happens for the Fisher-Rao metric tensor as it is noted in the previous reference and in this question that I asked, and answered, reading with the due care the previous reference); the von Neumann-Umegaki relative entropy leads to the so-called Bogoliubov-Kubo-Mori metric tensor. (All these three Riemannian metric tensors are related with group actions on $\mathcal{S}_{+}$ of suitable extensions of the unitary group as noted here.)
Consequently, different relative g-entropies lead to different Riemannian geometries on $\mathcal{S}_{+}$, while all f-divergences  lead to the Fisher-Rao Riemannian geometry on $\Delta_{+}$, and this is another big difference between the Information Geometry of $\Delta_{+}$ and $\mathcal{S}_{+}$.
Moreover, let me mention that even regarding dual connections there are incredible differences between the classical and quantum case. Indeed, it was noted here that it is not always true that the dual connections associated with a metric on $\mathcal{S}_{+}$ of the type classified by Petz are both torsion-free. This instance is something that is fascinating me a lot in this period, and I hope I will be able to understand it better in the future.
Finally, let me conclude by saying that, using von Neumann algebras (or C*-algebras), it is possible to formulate classical and quantum information geometry in a unified framework, and this could, hopefully, lead to a better understanding of the structural differences, and similarities, of these two subjects. In this work, a first attempt toward this goal is made for finite-level systems.
