# Tag Info

## Hot answers tagged markov-process

3

If $X$ is a random variable with continuous law, then for any real number $y$, $P(X = y) = 0$. If $X$ and $Y$ are independent random variables with continuous laws, then $P(X = Y) = 0$. To prove this, you can use the joint distribution and Fubini-Tonelli: \begin{align*} P(X=Y) &= \iint_{\mathbb R^2}1_{\{x = y\}}\mathcal L_X(dx)\mathcal L_Y(dy)\\ &= \...

3

The continuous case is quite analogous; it is simply $\pi_0 e^{tQ}$ (I think it is customary to put the law on the left and the functions on the right by adjointness). That is because the diffusion has to solve Kolmogorov's forward-backward equations. ($P(t) = e^{tQ}$ is your transition matrix for time $t$, and it satisfies $\frac{\partial P(t)}{\partial t} =... 1 You're close. Let$E_j$be the expected number of steps to get to state$j$from state$1$. Then $$E_j=E_{j-1}+\frac12(1)+\frac12(1+E_j)\implies E_j=2E_{j-1}+2$$ In order to get to state$j$, we must first get to state$j-1$. There's no reason to multiply by$\frac12$here. Then half the time, we finish in one more transition, and half the time we begin ... 1 The generator$Q$is for a continuus-time process. The Matrix$P(t)$is for a single discrete step of size$t$. They are related by the differential equation $$P'(t)=P(t)Q$$ with starting value$P(0)=I$(the identity matrix). In order to see why this equation makes sense you should imagine a small time-increment$\delta t$where you can assume that at most ... 1 You don't need to be told that$(B_t)$is Markov; this follows from the form of the covariance function. Why does it follow from$B_t=W_t-tW_1$,$0\le t\le 1$, that$(W_t)$is Gaussian, or even that$E[W_t]=0$? If$(W_t)$is one solution of$B_t=W_t-tW_1$,$0\le t\le 1$, then so is$\tilde W_t:=W_t+tZ$, where$Z$is any random variable on the probability ... 1 States 3 and 5 are the absorbing states of the Markov chain. The question is what is the probability of ending up in each state. There are some simplifications we can make. With probability 1, we will reach state 1. (Equivalently, the probability of never seeing a heads is zero.) If we are in state 2, we will eventually reach state 3 with probability 1. So,... 1 Two probability measures on a Polish space$E$coincide on$\mathcal B(E)$once their integrals assign the same values to each function in$C_b(E,\Bbb R)$, for example. Thus for each$x\in E$and each$t\ge 0$, the probability measure$P_t(x,\cdot)$is determined by$\bar P_t|_{C_b(E,\Bbb R)}$. And$C_0(E,\Bbb R)\$ is enough in the locally compact case.

Only top voted, non community-wiki answers of a minimum length are eligible