# Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

1,504 questions
0answers
6 views

1answer
20 views

### understanding Krylov-Bogolubov Theorem

Could anyone tell me what is $P(x,dy)$ and $P^{n+1}(x,dy)$ means here in 1p- 30, in the proof of thm 4.17? And, why $\phi$ was taken bounded by $1$? Are all $P^k(x,A)$, $P$, $Q^n$ probability measure ...
0answers
21 views

0answers
42 views
+50

1answer
17 views

### When doesn't a Markov Chain have a stationary distribution?

Is it possible for a Markov Chain not to have a stationary distribution? When doesn't a Markov Chain have a stationary distribution?
2answers
16 views

### test a Markov Matrix for a stationary distribution.

Consider a Markov chain with transition matrix $P = \begin{bmatrix} 1/2&1/2&0\\ 1/5&4/5&0\\ 0&0&1 \end{bmatrix}$ How many stationary distributions does this chain ...
0answers
11 views

0answers
38 views

### Where do I find exercises very similar to this one?

In my book Operational Research by author Hamdy A. Taha is this following exercise A home supply store can place orders for fridges at the start of each month for immediate delivery. A cost of $\$ ...
0answers
22 views

### Markov probability matrix and steady state solutions

I am currently doing a question and can seem to find where my problem lies with my final solution. The question is based on markov probability and steady state solutions, but the solution I get for ...
0answers
24 views

0answers
17 views

1answer
17 views

1answer
105 views

### I don't understand the proof of Corollary 4.8.7 in the book of Ethier and Kurtz

I'm trying to understand the proof of Corollary 8.7 of Chapter 4 in the book Markov Processes: Characterization and Convergence by Stewart N. Ethier, Thomas G. Kurtz. Here is the theorem and its proof:...