Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am using VOMC in order to implement a real-time system. The Loop of the system is the following: LOOP{

1)Get Input and Train VOMC according to it 2)Get output from Markov Chain

} With the above description I am want to show that in every iteration my Markov chain should be altered, at least the values of some transitions.

If for example I had a transition of A->B with probability of .6 and A->C with 0.4. This would mean that 6 out of 10 times I would choose A->B and 4 out of 10 A->C.

First question how do I determine the order of such occurrences? Randomly?

Second Question, if that chain is modified on every it iteration which means that I can have: fx ITERATION_2: A->B 0.5, A->C 0.5 ITERATION_3: A->B 0.33, A->C 0.66

How can I solve such a system?

Any Theoretical or Practical information would be very much appreciated!

PS I am aware of some concepts of Markov Chains but as you can see I lack of the theoretical background in solving.

share|cite|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.