Let $X\sim\mathcal N(0,A)$ , where $A\sim Exp(1)$. How do I recover the joint distribution for $Z=(X,A)$? Unfortunately there is no recipe for computing the joint distribution, just the other way around (from the joint distr. to the marginal ones). Would appreciate any help to find an Ansatz for this task.
 A: The model for a typical observation $X$ conditional on unknown parameter $\theta$ is $f(x|\theta)$. As a function of $\theta$, $f(x|\theta)$ is called likelihood.
The functional form of $f$ is fully specified up to a parameter $\theta$. The parameter $\theta$ is supported by the parameter space $\Theta$ and considered a random variable. The random variable $\theta$ has a distribution $\pi(\theta)$ that is called the prior. The distribution $h(x,\theta) =f(x|\theta)\pi(\theta)$ is called the joint distribution for $X$ and $\theta$. The joint distribution can also be factorized as
$$
h(x,\theta) =\pi(\theta|x)f(x)
$$ and the distribution $\pi(\theta|x)$ is called the posterior distribution for $\theta$, given $X=x$. The marginal distribution $m(x)$ can be obtained by integrating out $\theta$ from the joint distribution $h(x,\theta)$
$$
m(x)=\int_{\Theta}h(x,\theta)\mathrm d\theta=\int_{\Theta}f(x|\theta)\pi(\theta)\mathrm d\theta
$$
Therefore, the posterior $\pi(\theta|x)=\frac{h(x,\theta)}{m(x)}$ can be expressed as
$$
\pi(\theta|x)=\frac{f(x|\theta)\pi(\theta)}{\int_{\Theta}f(x|\theta)\pi(\theta)\mathrm d\theta}
$$
Assume that an observation, $X$ is normally distributed with mean $\mu=0$ and variance $\sigma^2=A$. The parameter of interest, $A$ has exponential distribution with parameter $\lambda=1$.
So yo have the Bayesian model $X|A\sim\mathcal N(0,A)$ and $A\sim\mathsf{Exp}(1)$.
The joint distribution has density
$$\begin{align}
h(x,A)=f(x|A)\pi(A)&=\frac{1}{\sqrt{2\pi A}}\mathrm e^{-\frac{x^2}{2A}}\mathrm e^{-A}=\frac{1}{\sqrt{2\pi A}}\mathrm e^{-\frac{x^2}{2A}-A}
=\pi(A|x)m(x)
\end{align}
$$
and $$
m(x)=\int_0^{\infty} \frac{1}{\sqrt{2\pi A}}\mathrm e^{-\frac{x^2}{2A}-A}\mathrm d A=\frac{\mathrm e^{-\sqrt{2}|x|}}{\sqrt{2}}
$$
that is $X\sim\mathsf{Laplace}\left(0,\frac{1}{\sqrt 2}\right)$ and
$$
A|X\sim \pi(A|x)=\frac{1}{\sqrt{\pi A}}\mathrm e^{-\frac{x^2}{2A}+\sqrt{2}|x|-A}
$$
