# Tagged Questions

21 views

### Show that this Markov chain is recurrent

So I have a Markov chain on the nonnegative integers such that, starting from $x$, the chain goes to $x+1$ with probability $p$, $0<p<1$, and goes to state $0$ with probability $1-p$. I'm ...
6 views

25 views

### Markov processes limiting probability questions

I am going over previous mock exams in preparation for an upcoming exam and am having problems with parts (ii) and (iv) and was looking for some guidance. For part (ii), my thinking was that the ...
17 views

15 views

### Policy Adjustment in Markov Decision Process

I was using MDP on my work to make optimal decision. I used discrete time, finite state MDP. I assumed that I will have an initial parameters, like the Reward/Cost, state transition probabilities and ...
52 views

### Prove that something is a Markov chain

Let $\xi_0, \xi_1, \xi_2, ...$be independent, identically distributed, integer valued random variables. Define $Y_n$ = max{$\xi_i: 0 \leq i \leq n$}. Show that $(Y_{n)n\geq0}$ is a Markov chain and ...
76 views

### Inverse of a regular stochastic matrix

Is it true that the inverse of a regular stochastic matrix is also regular? Are there any other interesting features that the inverse may have of a regular stochastic matrix? Hope someone could answer ...
63 views

### General birth and death process

hi i need some help to understand the following (from the general birth and death process).I'll give some context first , then i ask questions. Consider general birth and death process with birth ...
If we have an integer valued stochastic process, are these implications correct? independent increments $\rightarrow$ Markov property Markov property $\nrightarrow$ independent increments stationary ...