Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
This might be a very naive question.
Wikipedia describes an exponential family as a distribution
$f(x \mid \theta) = h(x)\exp( - \theta x - A(\theta))$
where $A(\theta) = \int h(x)\exp( - \theta x) \, dx$
I want to understand the role of the underlying measure or basically the $h(x)$ in the above equation.
For instance can any function of $\theta$ be written as $A(\theta)$ for some choice of $h(x)$.
Every probability density is with respect to a measure. The Lebesgue measure is the measure that assigns to every interval its length, so that the measure of the interval $(2,6) = \{ x : 2<x<6\}$ is $6-2=4$. The standard normal density function is $\displaystyle x\mapsto\frac 1 {\sqrt{2\pi}} e^{-x^2/2}$ and that is a density with respect to Lebesgue measure. In integrals $\text{“}dx\text{''}$ represents the Lebesgue measure, and that means $$ \int_{(2,6)} 13\,dx = \Big( 13\times\text{the measure of } (2,6) \Big) = 13\times 4 = 52. $$ If one multiplies the density $\displaystyle \frac 1 {\sqrt{2\pi}} e^{-x^2/2}$ with respect to Lebesgue measure by the Lebesgue measure $dx$ then one gets a measure called $\displaystyle \overbrace{\frac 1 {\sqrt{2\pi}} e^{-x^2/2} \, dx}^\text{This includes “$dx$''.}$, which is the standard normal distribution. The measure that that assigns to the interval $(2,6)$ is $$ \int_{(2,6)} \frac 1 {\sqrt{2\pi}} e^{-x^2/2} \, dx = \int_2^6 \frac 1 {\sqrt{2\pi}} e^{-x^2/2} \, dx, $$ the probability that a random variable with a standard normal distribution is in the interval $(2,6)$.
The measure considered in the definition of "exponential family" is not $h(x)$, but $h(x)\,dx$. And this is somewhat misleading, in that it suggests that there is always some density function $x\mapsto h(x)$ that gets integrated with respect to Lebesgue measure to get the measure $h(x)\,dx$. But in fact one often considers discrete measures, made up entirely of point masses, that assign positive measure only to sets of Lebesgue measure zero, so there is no density that can be integrated with respect to Lebesgue measure to obtain that discrete measure. For example \begin{align} x\mapsto {} & \binom n x p^x (1-p)^{n-x} = \binom n x \left( \frac p {1-p} \right)^x (1-p)^n \\[12pt] = {} & h(x) \exp\left( -x\theta - A(\theta) \right) \\ & \text{where }\theta = -\log \frac p {1-p} \text{ and }A(\theta) = -n\log(1-p) \text{ and }h(x) = \binom n x. \end{align} Here one integrates not with respect to Lebesgue measure but with respect to counting measure on the set $\{0,1,2,\ldots,n\}$, so that the measure of any set $A$ is the number of points in $A\cap\{0,\ldots,n\}$, and so the integral is a sum: $$ \int g(x) \, dc(x) = \sum_{x=0}^n g(x) \text{ where $c$ is the counting measure on } \{0,\ldots,n\}. $$
As to which functions can be realized as $\theta\mapsto A(\theta)$, that is a matter of inverting a Laplace transform. There are some inversion theorems, but I'm not sure if we need separate inversion theorems for different measures.
