# Questions tagged [information-geometry]

Information theory is a study of probability and statistics using the techniques of differential geometry. The area covers statistical model, Fisher metric, $\alpha$ connection, canonical divergence etc.

63 questions
Filter by
Sorted by
Tagged with
101 views

### What do the level sets of the Shannon entropy look like?

The Shannon entropy of a discrete probability distribution $\newcommand{\bs}{\boldsymbol{#1}}\bs p\equiv (p_i)_{i=1}^n$ is defined as $H(\bs p)\equiv -\sum_{i=1}^n p_i \log p_i$. Consider the ...
26 views

### How does curvature affect KL Divergence?

I have recently started studying Information Geometry and I came across this blog Natural Gradient Descent which talks about the Hessian acting as the curvature of KL divergence. I was wondering, is ...
30 views

### Counterexample to finiteness of KL divergence?

Suppose $P,Q$ are two probability distributions on a measurable space $(X, \mathcal F)$. The KL-divergence between $P$ and $Q$ is defined as \begin{equation} D_{KL}[P||Q] = \int_X \log \frac {dP}{dQ}...
10 views

138 views

### What is the connection between information monotonicity and the additivity property of information measures?

First of all, sorry if the question is trivial or not clear. I tried to be as clear as possible, but I'm a beginner in information geometry, with a background in physics, self-studying information ...
145 views

### Coordinate independent definition of Fisher metric on statistical manifolds

Is there a manifestly coordinate independent definition of the Fisher metric? I was reading Methods of Information Geometry by Amari and Nagaoka and Information Geometry and Its Applications by Amari, ...
177 views

### Basis vectors in information geometry

A basis vector in the tangent space at a point of a smooth manifold is given by a differential operator such as $\partial_{i}=\cfrac{\partial }{\partial \xi^{i}}$. On the other hand, in a statistical ...
544 views

### Pythagoras theorem holds with equality for linear families

The Pythagoras theorem for information projections is given by: $$D(p||q) \geq D(p||p^*) + D(p^*||q)\;\; \forall p,$$ where $p^* = argmin_{r \in {\cal S}} D(r||q)$, or the projection of the ...
164 views

### Wasserstein-like measure for disconnected spaces

Does there exist a Wasserstein-like measure for difference between probability distributions, where the space on which the probabilities are defined is not connected? E.g. a distribution over six ...
58 views

### the output dependency of DMC channel when the input components are independent with each other.

Most textbooks don't assert that the distributions of components of the output sequence $Y_{1},Y_{2},..Y_{n}$ of the discrete memoryless channel(DMC) are independent with each other even if the ...
165 views

154 views

In the Preface of 'Methods of Information Geometry': We consider the set, $S$, of normal distributions $P(x; \mu, \sigma) = \dfrac{1}{\sqrt{2\pi }\sigma}\exp \left \{ - \dfrac{(x-\mu)^2}{2\sigma^2}\... 1answer 1k views ### Self-studying Information Geometry I was recently exposed to the topic of Information Geometry by a friend of mine, and was looking for a good book to begin self-studying this topic. Any suggestions? Also, what subject matter would ... 1answer 712 views ### Higher math and statistics/probability So I've heard that certain areas of statistics and probability use manifolds and results from analysis and topology. Given that I lack the background to see where manifolds would become useful in ... 2answers 825 views ### reference request for a book on high dimensional probability and data analysis written for mathematicians I hope someone can help with this. I am a statistician looking for a good book on high dimensional probability and data analysis. Basically I am looking for the equivalent of Terry Tao's 2 volume set ... 1answer 464 views ### Limit of the multinomial distribution on the simplex Let$\Sigma_k$be the$k$-dimensional simplex$\{x_1,\dots x_{k+1}| \sum_j x_1=1\}$. Given a set of parameters$\vec{q}=(q_1, \dots q_{k})$in$\Sigma_{k-1}$and a bunch of non-negative integers$\vec{...
Given two probability distributions $P$ and $Q$ over the same outcome and event space (assume finite if needed) one defines their Renyi divergence as \$D_\alpha (P \vert \vert Q) = \frac{1}{\alpha -1} \...