# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

55 views

### Given two biased coins, the probability of obtaining heads on the $i^\text{{th}}$ toss using the following strategy?

We are given two coins: A and B with probability of obtaining heads being: $\alpha$ and $\beta$ respectively. The following sampling rule is used for i=1,2,...: If the $i^{\text{th}}$ toss results in ...
21 views

### Why does this MCMC algorithm to estimate parameters of a linear equation not converge to the posterior distribution?

As a kind of proof of principle I'm trying to estimate the parameters of a linear equation (before moving on to ODEs) using Markov Chain Monte Carlo sampling. The post that I am following can be found ...
91 views

### Showing a queueing system is a Markov Chain

I generally understand how to do this but I'm having trouble with a formal proof. "Consider an $M/M/1/m+1$ queue with exponential arrivals rate $\lambda$, exponential service rate $\mu$, and finite ...
119 views

293 views

### Gambler's Ruin and Markov Chains

Suppose that on each play of a certain game, a person will either win one dollar with the probability of $\frac{2}{3}$ or lose one dollar with probability $\frac{1}{3}$. Suppose also that the person's ...
57 views

### Markov DTMC and CTMC: How to build a $Q$ generating matrix using given $2$ state $q$ probabilities? Find Steady State Probabilities

I know that $$q_{(i,j)(i+1,j)}=\lambda_1$$ $$q_{(i,j)(i-1,j+1)}=\lambda_2$$ $$q_{(i,j)(i+1,j-1)}=0.5\lambda_3$$ $$q_{(i,j)(i,j-1)}=0.5\lambda_3$$ where $\lambda_1 , \lambda_2 , \lambda_3$ are rates ...
33 views