A simple example would be:
I have 3 states, A,B,C
The transition matrix for the chain is:
A B C A 0.1 0.3 0.6 B 0.4 0.2 0.4 C 0.5 0.2 0.3
and the transition cost matrix is:
A B C A 3 5 1 B 6 8 3 C 9 2 3
so, for each step, say, from A to C, the chance is 0.6, and the cost is 1
If the chain starts from A, and goes on 10 steps (so there are 11 states including A and 10 transitions), how to get the expected sum of the cost?
What's is this kind of problem called? I am able to get the expected number of visited states, but feel a bit confused as the cost is associated with transition but the states.