Consider $r_t = \mathbb{E}[Z_t]^T\beta$; $t= 1, 2, 3, \cdots$, where $r_t \in \mathbb{R}$, $\beta \in \mathbb{R}^d$ and $Z_t$ is a $d$-dimensional random vector.

Let's say $\beta$ is an unknown parameter and we would like to estimate it with a Bayesian approach by observing $r_t, Z_t$ for $t=1, 2, 3, \cdots$ (There is no iid assumption on random variables $Z_t$ and their distribution is unknown).How should we proceed? Any ideas?

My Approach: In the case that $d=1$, we may say $\mathbb{E}[Z_t] = \frac{1}{\beta}r_t$, and then say that assuming $\frac{1}{\beta} \sim f_\theta$ as a prior and $Z_t = \frac{1}{\beta}r_t$ we get a prior on $Z_t$. Then, we can update $\theta$ with posterior based on our observation $Z_t = z_t$. I have two questions here:

  • First, is my understanding correct in the case that $d=1$?
  • How can I extend it to $d > 1$?
  • $\begingroup$ This question seems too vague to deal with since nothing at all is known about $Z_1,Z_2,\cdots$. $\endgroup$ – Saad Apr 17 '18 at 11:37

These are just some ideas for a way to approach this.

Assuming that $r_t = E[Z_t]^T\beta = \beta^TE[Z_t]$, we can place $d(\mu_t, \sigma I_p)$ priors with some location and scale parameter on $Z_t$. If we place $N(\mu_t, \sigma I_p)$ priors we get some analytical tractability.

That is if $Z_t \sim \textrm{N}(\mu_t,\sigma^2 I_p)$, then $r_t \sim \textrm{N}(\beta^T\mu_t,\sigma^2\beta^T\beta)$, and we can place a prior on $\mu_t$, $\beta$, and $\sigma$. It's easier to assume that you at least know to what degree you're uncertain about your estimates $Z_t$ of $E[Z_t]$; meaning that $\sigma^2$ is a parameter to be specified.

In that setting, if $\mu_t \overset{iid}{\sim}\textrm{N}(0,\tau^2 I)$, where $\tau$ is a parameter to give a dispersed uninformative prior, and $\beta \sim \textrm{N}(0,\phi^2 I)$.

The likelihood for the model is then:

\begin{equation} \mathcal{L}(r_t | \mu_t, \beta) = \frac{1}{\sqrt{2\pi\sigma^2\beta^T\beta}}\exp\{-\frac{1}{\sigma^2\beta^T\beta}(r_t - \beta^T\mu_t)\} \end{equation}

The full posterior is then proportional to:

\begin{equation} \pi(\mu_t, \beta | r_t) \propto \frac{1}{\sqrt{2\pi\sigma^2\tau^2\phi^2 \beta^T\beta}}\exp\{-\frac{1}{\sigma^2\beta^T\beta}\sum_{t}(r_t - \beta^T\mu_t)^2\}\exp\{-\frac{1}{\tau^2}\beta^T\beta\}\prod_{t}(\exp\{-\frac{1}{\phi^2}\mu_t^T\mu_t\}) \end{equation}

Which seems like a pretty monstrous posterior to analyze, but should be fairly straightforward using Metropolis Hastings.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.