3
$\begingroup$

While the correct definition of exponential family is

$$ f_X(x\mid\theta) = h(x) \exp \left (\eta(\theta) \cdot T(x) -A(\theta)\right ), $$

it seems that in many materials I read, they don't pay much attention to $h(x)$. Sometimes, authors just drop this term.

Can somebody tell me more about the meaning of this $h(x)$ term? The most detailed description about this term I've found is "simply reflects the underlying measure w.r.t. which $p(x\mid\theta)$ is a density." (See http://stat-www.berkeley.edu/pub/users/mjwain/Fall2012_Stat241a/reader_ch8.pdf). Also, some people call it "base measure".

I kind of understand that $h(x)$ may not be important because it's trivial in most cases (1 or a constant like $1/\sqrt{2\pi}$), but for some distributions, e.g. Poisson, this term is nontrivial ($1/x!$), and it moderates the exponential term so greatly.

My understanding of this term is 1) it assigns a base probability to each element in the space of $x$, and the exponential term modifies this base probability. 2) it's here so that we don't get unbounded log-partition function $A(\theta)$, in cases like Poisson.

Can anybody tell me more about this $h(x)$, and any comment on my understanding of it is welcomed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.