# “Multivariate” Markov Chains

I am interested in estimating regime-switching VAR models to a regime setup I don't know the name of. I am hoping that someone can help me out with some references, or if there exists a name for it then tell me that so I know what to search for.

Suppose I have a vector of endogenous variables $\mathbf{y}_t$ and a vector of exogenous variables $\mathbf{x}_t$. Both vectors are observed. Including regime switching in the "usual" setting means that the conditional density of the endogenous vector is given by $$f(\mathbf{y_t}|s_t=j, \mathbf{x}_t, \mathscr{Y}_{t-1}; \boldsymbol{\alpha})$$ where $\mathscr{Y}_t=(\mathbf{y}_t, \dots, \mathbf{y}_1, \mathbf{x}_t, \dots, \mathbf{x}_1)$, $s_t$ is the state process such that it is currently in state $j$ and $\boldsymbol{\alpha}$ is a set of parameters for the conditional density.

What I've managed to find is a regime setup such that $s_t$ evolves according to a Markov chain satisfying $$P\{s_t=j|s_{t-1}=i, s_{t-2}=k, \dots, \mathbf{x}_t, \mathscr{Y}_t\}=P\{s_t=j|s_{t-1}=i\}.$$

However, I want to extend this --- suppose that there is another state process, $s_t^*$, which is the state process governing the exogenous vector $\mathbf{x}_t$. I want the $s_t$ process to evolve according to $$P\{s_t=j|s_{t-1}=i, s^*_{t-1}=k\}.$$

• Does this have a name? I've found some things that sound interesting (multivariate, composite, double, etc), but not sure they're what I'm after.
• Does anyone have any references to books and/or papers where such a setup is used?

I would really appreciate some help!

• If by $$P\{s_t=j|s_{t-1}=i, s_{t-2}=k, \dots, \mathbf{x}_t, \mathscr{Y}_t\}=P\{s_t=j|s_{t-1}=i\}.$$ you mean that your transition probability is independent of $\mathbf x_t$ this means that the transition probability is also independent of $s^*_t$, and $$P\{s_t=j|s_{t-1}=i, s^*_{t-1}=k\} = P\{s_t=j|s_{t-1}=i\}.$$ – Dimas Feb 5 '14 at 18:25
• @Dimas I think what I mean is that $s_t$ and $\mathbf{x}_t$ are independent only conditional on $s_t^*$. Do I make sense here? So given the "exogenous state" $s_t^*$, the $s_t$ probabilities change. Given only $\mathbf{x}_t$, that also changes the $s_t$ probabilities but only because $\mathbf{x}_t$ tells us something about $s_t^*$. It doesn't really matter whether it's $P\{s_t=j|s_{t-1}=i, s_{t-1}^*=k\}$ or $P\{s_t=j|s_{t-1}=i, s_{t}^*=k\}$. – hejseb Feb 5 '14 at 19:51

By what I understand, you can model your problem with a single larger Markov chain. Say the states of the first markov chain are $R=\{1,\dotsc,m\}$ and of the second are $S=\{1,\dotsc,n\}$. Then you can make a Markov chain taking values over the Cartesian product $X=R\times S$ whose state $x_t=(s_t, s_t^*)$ has the following transition densities:
$$P(s_t=i,s_k^*=k|s_{t-1}=j, s_{t-1}^*=\ell) = P(s_t=i | s_{t-1}=j, s_{t-1}^*=\ell)P(s_t^*=k | s_{t-1}^*=\ell).$$
Note that the second Markov chain, with state $s^*$, is independent of the first (which is not a true Markov chain as it has this weird exogenous input). This final chain is a regular Markov chain, though, and has no exogenous inputs.
• Thank you, that's what I've been looking at as well. I've found one paper, "Conditional Markov chain and its application in economic time series analysis" by Bai and Weng (2011), which has a similar formulation. I am not completely sure how this should be estimated, but hopefully that's possible to figure out. I think the main problem is my formulation of the setup/model, as I have $\mathbf{x}_t$ as exogenous but at the same time it's governed by a stochastic state process. In any case, I would like to avoid specifying a model for $\mathbf{x}_t$ (which I would need to obtain its pdf). – hejseb Feb 6 '14 at 7:28
• If you allow me to think out loud for a minute, here's another thought. Say we define $\tilde{s}_t=1$ to be the $(1, 1)$ outcome, $\tilde{s}_t=2$ to be $(1, 2)$ and so on. Does the estimation automatically 'understand', so to speak, that it's related to two chains? For example, say that $m$ and $n$ are both 2 so $\tilde{s}_t\in\{1, 2, 3, 4\}$. How is that different from simply starting out with a four-state $s_t$ and a non-existent $s_t^*$? Do you see what I mean? – hejseb Feb 6 '14 at 9:12
• Ok, thank you. I'll have to think about this some more to try to wrap my head around it. I think a side issue is that I want to have coefficient matrices that vary depending on the state of $s_t$, but they should not change when $s_t^*$ changes. So if I define a new chain as the combination of the two and estimate it the usual way, I'll get $m\times n$ coefficient matrices, but I only want $m$. So basically the model should change when $s_t$ changes, and the transition probabilities of $s_t^*$ should change when $s_t^*$ changes. But thanks for the help, you'll get the bounty when time's up. – hejseb Feb 7 '14 at 7:56