2
votes
Accepted
Showing that a martingale is constantly zero
Consider the Martingale $M_k:=W(Y_{k \wedge T})$, where $T$ is the first time $Y$ hits $i_0$, and $Y_0=i$. From $E[(M_k-M_0)^2]<\infty$, the Martingale is bounded in $L^2$, so it is uniformly ...
2
votes
Accepted
Search for a non absorbing Markov chain
Suppose $p(0,1)=p(1,0)=1$ and $p(2,3)=p(3,3)=1$.
This chain with 4 states has an absorbing state at state 3, but is not an absorbing chain because there is no way to get from state 1 to state 3.
...
1
vote
Some question about probability (i guess using Markov chains)
To simplify notation, let me call the initial probabilities $p' \; and \;q'$ for boy and father, $p'+q' <1$, and ignore draws (which don't matter)
P(boy wins a game) $=p=\frac{p'}{p'+q'}$
P(father ...
1
vote
Which State Will the Markov Chain Go To Next?
Define these events for convenience:
$A = (J_n = h)$
$B = (S_{n+1} > d)$
$C_j = (J_{n+1} = j)$
Conditioned on $J_n = h$ and $S_{n+1} > d$ (i.e. already waited $d$ time in $h$), the ...
Only top scored, non community-wiki answers of a minimum length are eligible
Related Tags
markov-chains × 5333probability × 2125
stochastic-processes × 1643
markov-process × 1244
probability-theory × 1159
random-walk × 397
linear-algebra × 302
matrices × 296
statistics × 294
probability-distributions × 194
transition-matrix × 176
expected-value × 126
queueing-theory × 106
measure-theory × 96
eigenvalues-eigenvectors × 88
reference-request × 87
monte-carlo × 86
graph-theory × 85
conditional-probability × 84
martingales × 80
stopping-times × 80
combinatorics × 68
ergodic-theory × 68
random-variables × 67
poisson-process × 66