# Tag Info

Elaborating on @Did's comment (letting $a_0(s)=\lambda s$ and $b_0(s)=1-\lambda$ as according to the initial distribution), we have the recurrences \begin{align} a_{n+1}(s) &= p_asa_n(s) + (1-p_b)sb_n(s),\\ b_{n+1}(s) &= (1-p_a)a_n(s) + p_b b_n(s). \end{align} Adding these yields $$a_{n+1}(s) + b_{n+1}(s) = (1-p_a+p_as)a_n(s) + (1-p_b+p_bs)b_n(s), ... 2 By Cauchy-Schwarz, if f\in \mathcal B_b(H) then (P_tf)^2\le P_t1\cdot P_t(f^2)\le P_t(f^2). If also f^2\in \mathcal C, then$$ \int_H (P_tf)^2\,d\pi\le\int_H P_t(f^2)\,d\pi=\int_H f^2\,d\pi<\infty. $$If follows that if \mathcal C is not only a vector space but also closed under multiplication (i.e., an algebra) then f\mapsto P_tf is a densely ... 2 It is the first visit.$$f_{ij}^{(n)} = P(X_n = j,\: X_k \not = j, \: k=1,2,\ldots,n-1 \mid X_0 = i)$$You can make two first revisits to 1 by n=2: \:\:1 \rightarrow 3 \rightarrow 1 n=3: \:\:1 \rightarrow 3 \rightarrow 2 \rightarrow 1$$f_{ij}^{(n)} = \sum_{k \not = j} p_{ik} f_{kj}^{(n-1)} 2 I think you misunderstood the meaning there. They talked about "contingent" claims, these are random variables given some other process. A simple example will be something like this: you have a simple SDE, dX = dB, with X_0 = a, the (strong) solution is, of course, X_t = a+ B_t. Now, you have another process on the same Brownian Filtration: dY ... 2 If X has generator G and b>0 is a scalar, then X(bt) is a Markov process with generator bG. If G_1 and G_2 are generators on the same state space, and (writing D(G_i) for the domain of G_i) D(G_1)\cap D(G_2) is large enough, then G_1+G_2 is the generator of another Markov process. Under the appropriate conditions, the semigroup S ... 2 A. This is pretty straight forward. Imagine T_k as the k-th hitting time of the state j, or the time when the Markov chain hits/reaches state j for the k-th time. Y_n is the time between n-1 and n-th hits. So Y_n=i means all the i-1 states between the n-1-th and n-th hits did not hit the state j, which implies all the states of X ... 1 Hint: Part (a) Let x_0, x_1, x_2 \in \left\{ -1,1\right\} \begin{align*} Pr\left(X_2=x_2|X_1=x_1, X_0=x_0\right) &=Pr\left(U_2U_3=x_2|U_1U_2=x_1, U_0U_1=x_0\right) \\ &=Pr\left(U_3=x_2U_2|U_2=x_1U_1, U_1=x_0U_0\right) \\ &=\frac{1}{2}Pr\left(U_3=x_2U_2|U_2=x_1U_1, U_1=x_0U_0,U_0=1\right)+\frac{1}{2}Pr\left(U_3=x_2U_2|U_2=x_1U_1, ... 1 You're certain to take exactly one step each from 1 to 2 and from 3 to 4; that's the constant 2. At 2, you have a Bernoulli experiment with success probability \frac23 that you perform until it succeeds, and at 4 you have a similar Bernoulli experiment with success probability \frac35. The expected time until a Bernoulli experiment succeeds ... 1 Here is an example in discrete time explaining why the Markov property is crucial to guarantee that solutions do not cross, in the sense that if two solutions coincide at some given time then they coincide at any later time. Consider the AR2 process (x_n)_{n\geqslant0} defined by some initial conditions (x_0,x_1) and by the recursion ... 1 We need to compute:S_N=\sum_{i=1}^{N}\binom{2N}{N+i}i \tag{1}$$but:$$ \sum_{i=1}^{N}\binom{2N}{N+i}N = \frac{N}{2}\left(4^N-\binom{2N}{N}\right)\tag{2}$$and:$$ \sum_{i=1}^{N}\binom{2N}{N+i}(N+i) = 2N\sum_{i=1}^{N}\binom{2N-1}{N-i}=\frac{N}{2}\cdot 4^N\tag{3}$$from the symmetry of binomial coefficients and the binomial formula. By (2) and (3) we ... 1 The statement is correct with X_\tau, but you have to read it carefully. The LHS is a conditional expectation, hence a random variable, so for some \omega \in \Omega you have that \mathbb{E}(f(X_{\tau+1}, \ldots) \mid \mathcal{F}_\tau) = \mathbb{E}_{X_{\tau}(\omega)}(f(X_0, \ldots)), i.e. you can restart your Markov chain at some random point and ... 1 If k\leq (n-1) you're done. Otherwise, k\geq n. You know that it's possible to reach state j from i in at most k steps with positive probability. By pigeon-hole principle, you have to visit at least one state twice in those k steps (since k\geq n). If you now delete the loop to that path, you've necessarily reduced the number of steps to a ... 1 I got the answer! Suppose there is no r \in \{1, \dots, n-1\} such that P^r(j, i) > 0. So r \geq n (we know there is some r that it happens). Then there are i_1, \dots i_r such that 0 < \leq P(j, i_1)P(i_1, i_2)\dots P(i_r, i). As r \geq n, there is some m_1 and m_2 such that i_{m_1} = i_{m_2}. Then you can "cut" the path ... 1 The markov property specifies that the probability of a state depends only on the probability of the previous state. You can "build more memory" into the states by using a higher order Markov model. In an nth order Markov model$$P(x_i | x_{i-1}, x_{i-2},..., x_1) = P(x_i | x_{i-1},..., x_{i-n} ) $$Example of a second order MC 1 There is nothing radically different about second order Markov chains: if P(x_i|x_{i-1},..,x_1)=P(x_i|x_{i-1},..,x_{i-n}) is a "n-th order Markov chain", we can still interpret it as a first order Markov chain, on the space of combinations of n states, i.e. S^n, if S is the set of values x_i takes: just write ... 1 Consider the following extension of \tau(1,2): At time \tau(1,2), choose a target state at random (with distribution \pi, independent of what has gone before) and continue until the first time that state is hit. The mean time for this total trip is \Bbb E[\tau(1,2)]+\sum_j \Bbb E_2[D_j]\pi_j=\Bbb E[\tau(1,2)]+K. Here D_j:=\inf\{n\ge 0: X_n=j\}, ... 1 Thank you so much for the explanation. Your first tip about the form of correlation function R(s,t) for Gaussian Markov processes was really helpful. Assume that we know that Gaussian process \xi_t with the correlation function R(s,t) is Markovian, then it implies that$$R(s,u)R(t,t)=R(s,t)R(t,u)$$as you told. I have no enough knowledge to prove it ... 1 1. It turns out that a centered Gaussian process is Markov if and only if its covariance function \Gamma: \mathbb{R}\times\mathbb{R} \to \mathbb{R} satisfies the equality:$$\Gamma(s,u)\Gamma(t,t)=\Gamma(s,t)\Gamma(t,u)\ \ \ \ (1)$$for all s<t<u. It turns out to be difficult to find a clear proof of the above fact; one that I found but did not ... 1 I don't know if these will help you or not, but it looks like you might want a bivariate distribution for your emission probabilities (one variable being the folding energy and the other parameter being the codons). Here are some references I found quickly regarding the construction of a joint distribution for which one variable is continuous and the other ... 1 If \tau_b\leqslant n, then S_j\geqslant b for some 1\leqslant j\leqslant n, and hence$$\max_{1\leqslant k\leqslant n}S_k\geqslant S_j\geqslant b. If $\max_{1\leqslant k\leqslant n}S_k\geqslant b$, then $\tau_b\leqslant \operatorname{argmax}_{1\leqslant k\leqslant n}S_k\leqslant n$. It follows that the two sets are equal.
$M/M/1$ and its relatives are the queues which are markov chains. This is since these queues have exponentially distributed interarrival times and service times. $M/D/1$ is not a markov chain but there exists an imbedded discrete time markov chain whose properties provide information about the process. $M/G/1, G/G/1$ etc are not markov chains. But if ...