Markov Chains: From Theory to Application A question that I have always wondered about is that how are the Transition Probabilities within a Markov Chain estimated in real-world applications?
I tried to learn more about this online and found the following link (https://hesim-dev.github.io/hesim/articles/mlogit.html). Over here, the following equation and explanation is provided:

Procedurally, I was trying to understand how this works.
Suppose there is a dataset with 3 States (State A, State B, State C). Each row in this dataset is an individual medical patient and contains some information on their covariates (e.g. height, weight, blood pressure, etc.), the state they were in and the state they transitioned to (let's assume that transitions can only take place at fixed discrete times, e.g. the first of January) :
  patient_id initial_state final_state   height age   weight
1          1             A           B 147.9283  49 85.03746
2          2             B           B 147.6188  50 98.37848
3          3             A           C 146.0570  51 87.79418
4          4             C           A 147.0269  56 86.38467
5          5             C           A 158.2545  47 83.81863

Suppose we want to fit a DISCRETE TIME MARKOV CHAIN to this data and estimate the transition probabilities - My understanding is the following:

*

*Isolate a subset of all rows where the initial state was State = State A


*Fit a Multinomial Logistic Regression to this subset of rows - doing this will provide you with general equations to calculate the probability of anyone within the population transitioning to any of the 3 States based on their covariate vector. This will also tell you the effect of different covariates on the transition probability and if these effects were statistically significant


*Repeat these two steps from the other two states (i.e. isolate the subset where initial state = State B, etc.) and fit a Multinomial Logistic Regression.


*In the end, you will have a 3 x 3 transition matrix which equations (as provided above) that estimate the transition probabilities based on a given vector of covariates
Based on these transition probabilities, you can now perform standard calculations as is done with Markov Chains - for example, given an initial probability distribution vector, what is the probability that this Markov Chain will be State B after "k" iterations?


*I am not sure if in this transition matrix I described above, transition probabilities within a given row will sum to 1?
Is my understanding of the above correct?
Thanks!
 A: The method you describe will produce a reasonable model choice. However, you should carefully measure the predictive accuracy of the resulting model before you trust it too much.
You could also imagine a less descriptive model: instead of multinomial regressions, compute just one overall transition probability from state $A$ to $B$, just as the fraction of people in the dataset who transitioned from $A$ to $B$. This way you'd end up with one overall transition matrix that applies to everyone, instead of computing different transition matrices for each person.
The right modeling choice depends on the size and quality of your dataset, plus on how well your model's assumptions match the real-world situation. In general, if your dataset size or quality or modeling assumptions are a little weaker then that would push you toward using a simpler model. I'd recommend building and testing the model you described, but it could also be good to build the simpler model I suggested so you can compare: how much more powerful is your model compared to the simple one on this dataset?
