# Markov chains with nonlinear predictor variables

I have data tracking about 25,000 individuals as each one moves through a Markov chain.

I want to know the shape of the relationship between a continuous 'covariate' (my independent variable of interest) and the transition intensities between the different states. The shape is really important for my question.

To find the shape of the relationship (linear, quadratic, etc.), I would imagine doing polynomial regression of expected hitting time versus the continuous covariate.

However, as I understand it, predictor covariates are typically fitted to Markov chains by applying a linear regression term to each transition intensity $\lambda_{ij}$. For instance, this paper "introduces covariables as a proportional factor in the baseline transition intensities":

$$\lambda_{ij}(\textbf{z})=\lambda_{ij}\textbf{e}^{\beta_{ij} \textbf{z}}$$

("where $\beta_{ij}$ is the vector of regression coefficients associated with the vector of covariables $\textbf{z}$ for the transition between states $i$ and $j$")

It doesn't seem impossible that there could be a nonlinear relationship between the predictor covariate and the transition intensities. Perhaps, for instance, a farm can rush its crops through the 'Markov chain' of crop stages really quickly at an optimum temperature (i.e. high transition rates), but lower temperatures and higher temperatures both have very low transition rates. In this case, plotting a linear relationship between the predictor covariate (temperature) and the transition rates would be inappropriate.

Is the way around this simply to apply different nonlinear transformations to my predictor covariate, treating each transformation as different predictors, and seeing which yields the maximum-likelihood fit? I can't see how this could solve the problem. Alternatively, are there methods for dealing with nonlinear predictors of a Markov chain process?

(I plan to work with the msm package in R, but there may be other packages more suitable to this question out there...?)

Many thanks if you can help!

• You can also try stats.stackexchange.com, if you don't get an answer here. – Andrew Oct 3 '16 at 16:35
• Thanks, that's a good idea. I'll see what happens here first! – Sprog Oct 3 '16 at 16:39