# Questions tagged [information-geometry]

Information theory is a study of probability and statistics using the techniques of differential geometry. The area covers statistical model, Fisher metric, $\alpha$ connection, canonical divergence etc.

46 questions
Filter by
Sorted by
Tagged with
13 views

### Measuring items repetition against known sampling space

I have a known sample space : S = {1,2,3,4,5,6,7,8,9,10} Then I have n subsets of the above set , like : m1={1,2,3}, m2={1,3,4,10},m3={1,2,4}, m5={3,4,6,7,8,9,10} ... I need a neat way to ...
39 views

### Differentiability in quadratic mean (Fisher information)

Let $\mathcal{P} = \{P_\theta = p_\theta \mu , \ \theta \in \Theta \subset \mathbb{R}^d \}$ a statistical model dominated by a $\sigma$-finite measure $\mu$. Recall the definition of ...
61 views

### Statistical distance induced by Fisher information metric on statistical manifold of categorical distribution (n-dimensional simplex)

I am trying to compute the information length or distance induced by the Fisher information metric on the statistical manifold of the categorical distribution (the interior of the n-dimensional ...
19 views

### Relationship between Riemann metric and divergence in information geometry

I would like to ask you about formula(6) in this paper. Some people say that the Primary term of the divergence $D$ disappear. But how should I show that?
30 views

### upper bound on cross entropy or relative entropy

Am looking for upper bounds for relative-entropy or cross-entropy for non-gaussian case . All I am aware of is bounds for entropy based on determinant of covariance matrix. What about for relative ...
31 views

### Simple question on parallel transport in dually flat manifolds

I just started studying Information Geometry and its applications by Amari. Right in the first chapter, the author talks about parallel transport in Dually flat manifolds. Just some quick notation: ...
38 views

### A basic question on mutually orthogonal coordinate systems

I am reading the first chapter of Information Geometry and its applications by Amari. I am struggling to grasp a basic concept about mutually orthogonal coordinate systems. Since the book is not ...
16 views

49 views

### Find parameter transformation of probability distribution such that the transformed parameters are orthogonal

I'm looking for a parameter transformation of a probability distribution such that the resulting parameters are orthogonal. That is, the off-diagonal elements of the Fisher Information matrix of the ...
19 views

### local geometry interpretation of binary hypothesis testing

In information geometry, suppose $$\mathcal{N}_{\epsilon}^{\mathcal{X}}\triangleq\left\{P\in \mathcal{P}^{\mathcal{X}}:\sum_{x\in\mathcal{X}}\frac{(P(x)-P_0(x)^2}{P_0(x)}\leq \epsilon^2\right\}$$ ...
45 views

### Questions about the definition of a manifold [closed]

Amari's definition of an $n$-dimensional manifold [source] is: An $n$-dimensional manifold $M$ is a set of points such that each point has $n$-dimensional extensions in its neighborhood. That is, ...
75 views

### Maximizing a mutual information w.r.t. (i.i.d.) variation of the channel.

Intuitively, I have I have a discrete (finite) random variable $Z$ whose probability mass function $q(z)$ I can choose. $Z$ selects the conditional probability distribution $p(y|x,z)$ of another (...
53 views

352 views

650 views

### Self-studying Information Geometry

I was recently exposed to the topic of Information Geometry by a friend of mine, and was looking for a good book to begin self-studying this topic. Any suggestions? Also, what subject matter would ...
543 views

### Higher math and statistics/probability

So I've heard that certain areas of statistics and probability use manifolds and results from analysis and topology. Given that I lack the background to see where manifolds would become useful in ...
500 views

### reference request for a book on high dimensional probability and data analysis written for mathematicians

I hope someone can help with this. I am a statistician looking for a good book on high dimensional probability and data analysis. Basically I am looking for the equivalent of Terry Tao's 2 volume set ...
355 views

599 views

### Manifolds with volume forms on every submanifold

If we equip a manifold with an inner product (i.e. we have a Riemannian Manifold) then we get a canonical volume form on that manifold (please mentally insert the prefix "pseudo" into my question ...
943 views

### Determinant of Fisher information

In information geometry, the determinant of the Fisher information matrix is a natural volume form on a statistical manifold, so it has a nice geometrical interpretation. But what is it in ...
482 views

183 views

### Parallel translation via $e$-connection

This question is concerned with Section 2.5. of Amari and Nagaoka's Information geometry book. Let me give some background first. Let $\mathcal{P}$ be the $n$-dimensional manifold of all (strictly ...
Having some knowledge in differential geometry on $\mathbb{R}^n$, I'm reading a book on Information geometry by Amari. Let $S=\{p_{\theta}\}$, $\theta=(\theta_1,\dots,\theta_n)$ be an $n$-dimensional ...