Questions tagged [information-geometry]

Information theory is a study of probability and statistics using the techniques of differential geometry. The area covers statistical model, Fisher metric, $\alpha$ connection, canonical divergence etc.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Measuring items repetition against known sampling space

I have a known sample space : S = {1,2,3,4,5,6,7,8,9,10} Then I have n subsets of the above set , like : m1={1,2,3}, m2={1,3,4,10},m3={1,2,4}, m5={3,4,6,7,8,9,10} ... I need a neat way to ...
1
vote
0answers
39 views

Differentiability in quadratic mean (Fisher information)

Let $\mathcal{P} = \{P_\theta = p_\theta \mu , \ \theta \in \Theta \subset \mathbb{R}^d \}$ a statistical model dominated by a $\sigma$-finite measure $\mu $. Recall the definition of ...
2
votes
1answer
61 views

Statistical distance induced by Fisher information metric on statistical manifold of categorical distribution (n-dimensional simplex)

I am trying to compute the information length or distance induced by the Fisher information metric on the statistical manifold of the categorical distribution (the interior of the n-dimensional ...
0
votes
0answers
19 views

Relationship between Riemann metric and divergence in information geometry

I would like to ask you about formula(6) in this paper. Some people say that the Primary term of the divergence $D$ disappear. But how should I show that?
0
votes
1answer
30 views

upper bound on cross entropy or relative entropy

Am looking for upper bounds for relative-entropy or cross-entropy for non-gaussian case . All I am aware of is bounds for entropy based on determinant of covariance matrix. What about for relative ...
1
vote
1answer
31 views

Simple question on parallel transport in dually flat manifolds

I just started studying Information Geometry and its applications by Amari. Right in the first chapter, the author talks about parallel transport in Dually flat manifolds. Just some quick notation: ...
0
votes
1answer
38 views

A basic question on mutually orthogonal coordinate systems

I am reading the first chapter of Information Geometry and its applications by Amari. I am struggling to grasp a basic concept about mutually orthogonal coordinate systems. Since the book is not ...
0
votes
0answers
16 views

Simple criterion for exponential family

In probability theory, a parameterized distribution belongs to the exponential family if it can be written in the form $$ p(\mathbf{x}; \boldsymbol \theta) = h(\mathbf{x}) \exp\Big(\boldsymbol\eta({\...
0
votes
1answer
53 views

Exponential map on the Fisher manifold for exponential family distribution

So, I don't really understand too well Diff. Geometry and Manifolds currently. Hence, I've started studying it and it is very interesting. However, atm I just need to understand how to compute the ...
1
vote
1answer
134 views

Is the quantum relative entropy a Bregman divergence?

This is related to a broader question about quantum information geometry - this question is more specific. A fundamental concept in Amari's treatment of information geometry is that of a Bregman ...
14
votes
0answers
255 views

What is the essential difference between classical and quantum information geometry?

This question may be a little subjective, but I would like to understand, from a geometric perspective, how the structure of quantum theory differs from that of classical probability theory. I have a ...
1
vote
1answer
124 views

Connections and applications of/between information (field) theory and information geometry

I hope this isn't any kind of duplicate. I have looked through the forum, but it's possible I overlooked something. I'm currently in the 4th semester of my physics studies and listened to some ...
1
vote
0answers
52 views

$\alpha$ pythagorean theorem for $\alpha$ divergence in the probability simplex $S_n$

I'm trying to understand "Methods of information geometry" of Amari and Nagaoka, p72. Consider the probability simplex $$S_n:=\{p=(p_1,\cdots,p_{n+1})\in \mathbb{R}^{n+1}~|~\sum_i^{n+1} p_i=1,~p_i\...
0
votes
1answer
49 views

Find parameter transformation of probability distribution such that the transformed parameters are orthogonal

I'm looking for a parameter transformation of a probability distribution such that the resulting parameters are orthogonal. That is, the off-diagonal elements of the Fisher Information matrix of the ...
2
votes
1answer
19 views

local geometry interpretation of binary hypothesis testing

In information geometry, suppose $$ \mathcal{N}_{\epsilon}^{\mathcal{X}}\triangleq\left\{P\in \mathcal{P}^{\mathcal{X}}:\sum_{x\in\mathcal{X}}\frac{(P(x)-P_0(x)^2}{P_0(x)}\leq \epsilon^2\right\} $$ ...
1
vote
0answers
45 views

Questions about the definition of a manifold [closed]

Amari's definition of an $n$-dimensional manifold [source] is: An $n$-dimensional manifold $M$ is a set of points such that each point has $n$-dimensional extensions in its neighborhood. That is, ...
1
vote
0answers
75 views

Maximizing a mutual information w.r.t. (i.i.d.) variation of the channel.

Intuitively, I have I have a discrete (finite) random variable $Z$ whose probability mass function $q(z)$ I can choose. $Z$ selects the conditional probability distribution $p(y|x,z)$ of another (...
1
vote
1answer
53 views

Do the set of all factorisable distributions form an exponential family?

A (natural) exponential family of probability distributions is defined as $$ p(x|\theta) = \exp\big(g(x) + \theta\cdot f(x) - \psi(\theta)\big),\tag{1} $$ where $\theta$ is a vector of parameters and $...
2
votes
0answers
125 views

What is the connection between information monotonicity and the additivity property of information measures?

First of all, sorry if the question is trivial or not clear. I tried to be as clear as possible, but I'm a beginner in information geometry, with a background in physics, self-studying information ...
2
votes
0answers
130 views

Coordinate independent definition of Fisher metric on statistical manifolds

Is there a manifestly coordinate independent definition of the Fisher metric? I was reading Methods of Information Geometry by Amari and Nagaoka and Information Geometry and Its Applications by Amari, ...
2
votes
1answer
146 views

Basis vectors in information geometry

A basis vector in the tangent space at a point of a smooth manifold is given by a differential operator such as $\partial_{i}=\cfrac{\partial }{\partial \xi^{i}}$. On the other hand, in a statistical ...
3
votes
1answer
303 views

Pythagoras theorem holds with equality for linear families

The Pythagoras theorem for information projections is given by: $$D(p||q) \geq D(p||p^*) + D(p^*||q)\;\; \forall p,$$ where $p^* = argmin_{r \in {\cal S}} D(r||q)$, or the projection of the ...
3
votes
1answer
109 views

Wasserstein-like measure for disconnected spaces

Does there exist a Wasserstein-like measure for difference between probability distributions, where the space on which the probabilities are defined is not connected? E.g. a distribution over six ...
-1
votes
1answer
54 views

the output dependency of DMC channel when the input components are independent with each other.

Most textbooks don't assert that the distributions of components of the output sequence $Y_{1},Y_{2},..Y_{n}$ of the discrete memoryless channel(DMC) are independent with each other even if the ...
1
vote
0answers
139 views

Monotonicity of the Jensen-Shannon divergence

Let $\text{JSD}(P\mid\mid Q)$ be the Jensen-Shannon divergence (https://en.wikipedia.org/wiki/Jensen-Shannon_divergence) between two probability distributions $P$ and $Q$. Question: Is $\text{JSD}(P\...
6
votes
1answer
352 views

How many types of mathematical information are there?

I am aware of at least two types of mathematical information: Shannon Information, which is the negative of entropy (i.e. a loss of entropy by $n$ bits is precisely a gain of Shannon information of $...
1
vote
1answer
127 views

Question on the motivation of Information Geometry

In the Preface of 'Methods of Information Geometry': We consider the set, $S$, of normal distributions $P(x; \mu, \sigma) = \dfrac{1}{\sqrt{2\pi }\sigma}\exp \left \{ - \dfrac{(x-\mu)^2}{2\sigma^2}\...
10
votes
1answer
650 views

Self-studying Information Geometry

I was recently exposed to the topic of Information Geometry by a friend of mine, and was looking for a good book to begin self-studying this topic. Any suggestions? Also, what subject matter would ...
10
votes
1answer
543 views

Higher math and statistics/probability

So I've heard that certain areas of statistics and probability use manifolds and results from analysis and topology. Given that I lack the background to see where manifolds would become useful in ...
6
votes
2answers
500 views

reference request for a book on high dimensional probability and data analysis written for mathematicians

I hope someone can help with this. I am a statistician looking for a good book on high dimensional probability and data analysis. Basically I am looking for the equivalent of Terry Tao's 2 volume set ...
2
votes
1answer
355 views

Limit of the multinomial distribution on the simplex

Let $\Sigma_k$ be the $k$-dimensional simplex $\{x_1,\dots x_{k+1}| \sum_j x_1=1\}$. Given a set of parameters $\vec{q}=(q_1, \dots q_{k})$ in $\Sigma_{k-1}$ and a bunch of non-negative integers $\vec{...
0
votes
1answer
78 views

What modification is this of the notion of Renyi divergence?

Given two probability distributions $P$ and $Q$ over the same outcome and event space (assume finite if needed) one defines their Renyi divergence as $D_\alpha (P \vert \vert Q) = \frac{1}{\alpha -1} \...
20
votes
1answer
599 views

Manifolds with volume forms on every submanifold

If we equip a manifold with an inner product (i.e. we have a Riemannian Manifold) then we get a canonical volume form on that manifold (please mentally insert the prefix "pseudo" into my question ...
5
votes
2answers
943 views

Determinant of Fisher information

In information geometry, the determinant of the Fisher information matrix is a natural volume form on a statistical manifold, so it has a nice geometrical interpretation. But what is it in ...
8
votes
0answers
482 views

Geodesic equation from Christoffel symbols

Let $\mathcal{P}:=\mathcal{P}(\mathcal{X})$ be the $n$-dimensional manifold of all (strictly positive) probability vectors (distributions) on $\mathcal{X}=\{x_0,\dots,x_n\}$, i.e., each $p=(p(x_0),\...
1
vote
0answers
218 views

Geodesic question

Let $\mathcal{P}:=\mathcal{P}(\mathcal{X})$ be the $n$-dimensional manifold of all (strictly positive) probability distributions on $\mathcal{X}=\{x_0,\dots,x_n\}$. Each $p=(p(x_0),\dots,p(x_n))\in \...
2
votes
1answer
183 views

Parallel translation via $e$-connection

This question is concerned with Section 2.5. of Amari and Nagaoka's Information geometry book. Let me give some background first. Let $\mathcal{P}$ be the $n$-dimensional manifold of all (strictly ...
2
votes
1answer
233 views

Gradient of a function and inverse of metric

Having some knowledge in differential geometry on $\mathbb{R}^n$, I'm reading a book on Information geometry by Amari. Let $S=\{p_{\theta}\}$, $\theta=(\theta_1,\dots,\theta_n)$ be an $n$-dimensional ...
2
votes
0answers
153 views

Upper bound of mutual information in a Markov chain

Consider binary random variables $X$ and $V$ with marginal distributions $p$ and $\pi$ respectively and also the conditional distribution $p(X=x\mid V=v)=q(x\mid v)$, where $x\in\{-b,b\}$ and $v\in\{-...
1
vote
0answers
48 views

Independent transformation of probability measures

I have a pair of dependent random variable $(\theta, X)$ where $\theta\in K$ for a compact subset $K\subset\mathbb{R}$ and $X\in\mathbb{R}^d$ with marginals $P_{\theta}$ and $P_X$. I want to ...
2
votes
0answers
60 views

Sufficient statistics and isometries

Let $(M,g)$ be an infinite dimensional statistical manifold with the Fisher information metric $g$. Is it true that any isometry on this manifold must correspond to a sufficient statistic?
4
votes
0answers
220 views

What are the Big Theorems in Information Geometry?

I am working on preparing a talk on information geometry to a young finance/applied math audience. Motivating this area is turning out to be a little difficult. What are some big theorems or results ...
21
votes
3answers
3k views

Applications of information geometry to the natural sciences

I am contemplating undergraduate thesis topics, and am searching for a topic that combines my favorite areas of analysis, differential geometry, graph theory, and probability, and that also has (...
1
vote
2answers
768 views

Is a Riemannian metric positive definite or positive semidefinite?

From Wikipedia The Fisher information matrix is a N x N positive semidefinite symmetric matrix, defining a Riemannian metric on the N-dimensional parameter space, But a Riemannian metric is ...
2
votes
2answers
7k views

Get a Fisher information matrix for linear model with the normal distribution for measurement error?

For given linear model $y = x \beta + \epsilon$, where $\beta$ is a $p$-dimentional column vector, and $\epsilon$ is a measurement error that follows a normal distribution, a FIM is a $p \times p$ ...
3
votes
1answer
457 views

Rao-Cramer lower bound for Rician distribution

I derived ML estimator for Rician distributed data and I am trying to show Rao-Cramer lower bound of $\hat{A}$ estimator variance. $$f(x_k|A,\sigma) = \frac{x_k}{\sigma^2}\exp\left(-\frac{x^2_k+A^2}{...