Define a one-parameter exponential family as a family of densities of the form $$f_\theta(x)=\exp(\eta(\theta)T(x) + \xi(\theta))h(x)$$ where $T(x)$ and $h(x)$ are Borel functions, $\theta\in\Theta\subset\mathbb R$ and $\eta$ and $\xi$ are real-valued functions defined on $\Theta$.

Double exponential distribution is a distribution having the density $$p_\theta(x)= \frac{1}{2}\exp(-|x - \theta|)$$ for $\theta\in\mathbb R$.

I am looking for a simple proof of the theorem in the title. I found a proof in the book of Shao "Mathematical Statistics. Exercises and Solutions." but it uses a more general definition of exponential families and doesn't show why the classes are not compatible. What is the special feature of $p_\theta(x)$ that makes the representation as exponential family impossible?

  • $\begingroup$ The only additional generality assumed in Shao is that the distribution could be from a multiparameter exponential family. In order to prove the statement in your title, you have to show that the double exponential is not in the exponential family for all possible (finite) choices of the dimension of the parameter space. If you want a proof that it's not from a one-parameter family, then just collapse Shao's proof down to the $p = 1$ case. It still holds. :) $\endgroup$ – cardinal Nov 14 '11 at 0:09
  • $\begingroup$ The main feature is that you can't "factor" $|x-\theta|$ into a product of functions, one that depends only on $x$ and the other that depends only on $\theta$ (plus perhaps an additional function of only $\theta$. $\endgroup$ – cardinal Nov 14 '11 at 0:10
  • $\begingroup$ @cardinal Thanks for your comments! I should have mentioned I'm interested in a special case with one-dimensional $\xi$ and $T$. $\endgroup$ – Julian Wergieluk Nov 14 '11 at 2:28

Assume that $\Theta$ has at least three distinct points $\theta_1$, $\theta_2,$ and $\theta_3$. Suppose that $$\exp(\eta(\theta)T(x)+\xi(\theta)) h(x)={1\over 2}\exp(-|x-\theta|).$$ Since $h(x)$ never takes the value zero, we can write it as $h(x)={1\over 2}\exp(w(x))$, and deduce that, for all $x\in\mathbb{R}$ and $\theta\in\Theta$, $$\eta(\theta)T(x)+\xi(\theta) +w(x)= -|x-\theta|.$$

Substitute $\theta_1, \theta_2$ and subtract the two equations to get $$[\eta(\theta_1)-\eta(\theta_2)]\ T(x)+\xi(\theta_1)- \xi(\theta_2) = |x-\theta_2|-|x-\theta_1|.$$ Since the right hand side is not a constant function of $x$, we find that $\eta(\theta_1)\neq\eta(\theta_2)$ and hence that $T$ is differentiable in $x$, except possibly at $\theta_1$ and $\theta_2$. The same argument using the pairs $\{\theta_1 ,\theta_3\}$ and $\{\theta_2 ,\theta_3\}$ shows that $T$ is, in fact, differentiable everywhere.

We conclude that $|x-\theta_2|-|x-\theta_1|$ is everywhere differentiable in $x$, which is a contradiction.

  • 1
    $\begingroup$ (+1) I think this works, though it still only allows one to conclude that the double-exponential is not from a one-parameter exponential family. $\endgroup$ – cardinal Nov 14 '11 at 0:54
  • $\begingroup$ @cardinal Thanks. I'm not sure what $|x-\theta|$ would mean if $\theta$ were multidimensional. Or are you thinking of parametrizing by something other than $\theta$? $\endgroup$ – user940 Nov 14 '11 at 0:58
  • 1
    $\begingroup$ I mean that, conceivably, there could be functions $\eta : \mathbb R \to \mathbb R^p$ and $T: \mathbb R \to \mathbb R^p$ such that $\exp( \eta(\theta) \cdot T(x) + \xi(\theta) ) h(x) = \frac{1}{2} \exp(-|x - \theta|)$, in which case at least a curved exponential family would result. $\endgroup$ – cardinal Nov 14 '11 at 1:02
  • $\begingroup$ @cardinal Oh, I see. Thanks for the clarification! $\endgroup$ – user940 Nov 14 '11 at 1:05
  • $\begingroup$ Actually my intention was to find a simple proof in case of one-parameter exponential family with restriction $\eta:\mathbb R \to \mathbb R$. Thanks!! $\endgroup$ – Julian Wergieluk Nov 14 '11 at 2:08

Meanwhile I figured out another proof. But the one of Byron is clearly more elegant.

Following the idea of Shao we consider the quotients $\frac{p_\theta(x)}{p_{-\theta}(x)}= \frac{f_\theta(x)}{f_{-\theta}(x)}$. This allows us to get rid of $h(x)$ and yields \begin{equation} |x+\theta| - |x-\theta| = \left( \eta(\theta) - \eta(-\theta) \right)T(x) - \left( \xi(\theta) - \xi(-\theta) \right) \end{equation} Since $\eta(\theta) - \eta(-\theta)$ must be non-zero for some $\theta$ we define \begin{equation} A = \eta(\theta) - \eta(-\theta) \end{equation} \begin{equation} B =\xi(\theta) - \xi(-\theta) \end{equation} and get $ |x+\theta| - |x-\theta| = A T(x) + B $. Moreover $|x+\theta| - |x-\theta|$ is an antisymmetric function and therefore $A T(x) + B = -(A T(-x) + B)$. This yields $T(x) + \frac{B}{A} = - T(-x) - \frac{B}{A}$ and implies that, $T(x)$ shifted by $\frac{B}{A}$ is antisymmetric.

On the other hand, if we consider $\frac{p_\theta(x)}{p_{0}(x)}= \frac{f_\theta(x)}{f_{0}(x)}$ we get \begin{equation} |x| - |x-\theta| = \left( \eta(\theta) - \eta(0) \right)T(x) - \left( \xi(\theta) - \xi(0) \right) \end{equation} For positive $\theta$ this function is constant in $x$ for $x<0$. But $T(x)$ is also antisymmetric. This implies $T(x) \equiv T$ is constant for all $x\in\mathbb R$. A contradiction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.