# Tag Info

9

The probability of any chair being occupied can be calculated exactly and, for the third chair as asked for in the question, it comes to $513/1943=0.26402\ldots$. More generally, let $\lambda$ be the rate at which customers arrive multiplied by the time taken for a haircut. So, here, we have $\lambda=(10{\rm mins})^{-1}(15{\rm mins})=3/2$. Then, after a long ...

6

The difference between Markov chains and Markov processes is in the index set, chains have a discrete time, processes have (usually) continous. Random variables are much like Guinea pigs, neither a pig, nor from Guinea. Random variables are functions (which are deterministic by definition). They are defined on probability space which most often denotes all ...

5

This is not a solution, but it might be helpful, and it is too long for a comment. Your $N^N$ equations can be simplified, because you can exploit symmetry in the structure of the problem. Say that $L(S)$ is the expected number of rounds for the game to end after reaching state $S$, where $S$ is some string of length $N$. Then: $$\begin{eqnarray} ... 4 I can improve a little on MJD's method. This was based on computing for each state S (the last N values) the expected remaining number of steps L(S) until a final state (last N values equal) is reached. Let me change the notation slightly. Let T_S be the number of steps from state S until a final state is reached, and assume that this final ... 4 Suppose that for t\geq s, we have some function g so that$$E[f(X_{t}) \,\vert\, \mathcal{F}_{s}]=g(X_{s}) \quad \mbox{a.s.}\tag1$$Conditioning on X_s in (1) gives$$E[f(X_{t}) \,\vert\, X_s]=g(X_{s}) \quad \mbox{a.s.}\tag2$$From this we deduce$$E[f(X_{t}) \,\vert\, \mathcal{F}_{s}]=E[f(X_{t}) \,\vert\, X_s] \quad \mbox{a.s.}\tag3 $$Equation (3) ... 4 This has nothing to do with being a Lévy process or even with randomness. Assume that the function f:t\mapsto f(t) has no positive jump. Let t_y=\inf\{t\gt0\mid f(t)\gt y\}. Assume that t_y is finite and that y\gt f(0). Then f(t_y-h)\leqslant y for every h\gt0, by definition of t_y, hence \limsup\limits_{s\to t,s\lt t}f(s)\leqslant y. ... 4 Queueing theory uses Kendall's notation, as you described. There are three components describing the behavior of a queue: The customers arriving for service, which is usually described by a Poisson process (random arrivals), but sometimes by non-Poisson processes or even deterministic arrivals rates The time required to service each customer, which is ... 4 Note that both (Z\circ \theta_t)\, V and  \mathbb{E}_{Y_t}[Z]\, V are product functions whose expectation can be found using your main equation. Define$$\phi(z)=\mathbb{E}_z(Z)=g_0(z)\int K_{s_1}(z,dy_1)g_1(y_1)\cdots\int K_{s_m-s_{m-1}}(y_{m-1},dy_m)g_m(y_m).$$Then we calculate$$ \begin{eqnarray*} ...

4

Let $f: [0,\infty) \to [0,\infty)$ and $s,t \in [0,\infty)$. Then $$\DeclareMathOperator{cov}{cov} \cov(B_{f(s)},B_{f(t)}) = \min \{f(s),f(t)\}$$ since $\cov(B_u,B_v)=\min\{u,v\}$ holds for all $u,v \in [0,\infty)$, so in particular for $u:=f(s)$, $v:=f(t)$. Concerning your second question: Theorem: Let $(X_t)_{t \geq 0}$ a continuous local ...

3

Both answers are wrong... The system of three equations can be readily rewritten as $A=2B=C$ hence the only solution such that $A+B+C=1$ is $(A,B,C)=(\frac25,\frac15,\frac25)$. If $(A,B,C)=(\frac14,\frac12,\frac14)$ (your solution), then $A\ne2B\ne C$. If $(A,B,C)=(\frac15,\frac25,\frac15)$ (their solution), then $A+B+C\ne1$.

3

Usually, people deal with a right stochastic matrix i.e. each row contains non-negative real numbers and sum to $1$. The instructor on the video uses a left stochastic matrix i.e. each column contains non-negative real numbers and sum to $1$. Following the convention the instructor follows, note that for a $n \times n$ left stochastic matrix, we have ...

3

We will use the result if $\lim_{n\to \infty} \frac{a_{n+1}}{a_n}=a$ and $|a|<1$, then $\lim_{n\to \infty}{a_n}=0$. Let $$a_n = \binom {2n}{n} p^n(1-p)^n \implies \frac{a_{n+1}}{a_n}= {\frac { 2p(1-p)\left( 2\,n+1 \right) }{(n+1)}}.$$ Taking the limit of the last expression $$\lim_{n\to \infty} \frac{a_{n+1}}{a_n}= 4 p(1-p).$$ Now, note ...

3

We can show that ${2n \choose n}p^n(1-p)^n \rightarrow 0$ as $n \rightarrow \infty$ by the ratio test. Let $a_n = {2n \choose n}p^n(1-p)^n$, then $$\lim_{n \rightarrow \infty}\frac{a_{n+1}}{a_n} = \lim_{n \rightarrow \infty} p(1-p) \frac{(2n+2)(2n+1)} {(n+1)^2} = 4p(1-p)$$ Since $p$ is a probability, $4p(1-p) \leqslant 1$, if $p \neq \frac{1}{2}$, we're ...

3

First of all, it would be nice if you link the definitions of F-K formula and KBE you have in mind. Anyway: KBE is an equation used to study $P_t f(x):=\mathsf E_x[f(X_t)]$ in terms of the generator $$\mathcal A :=\lim_{t\downarrow 0}\frac{P_t - P_0}{t}.$$ This is indeed applies to general Markov processes, as its definition depends only such notions ...

3

Indeed $\sigma_x$ is exponentially distributed under $P_x$. To see this, note that $[\sigma_x\gt s]\subseteq[X_s=x]$ hence $$P_x[\sigma_x\gt t+s\mid\sigma_x\gt s]=P_x[\sigma_x\gt t+s\mid X_s=x]=P_x[\sigma_x\gt t].$$ This is specific to the starting distribution $P_x$. For other starting distributions, the distribution of $\sigma_x$ is the barycenter of ...

3

Since $X_s$ is $\mathcal F_s$-measurable, the tower property shows that $$\mathrm{P}_x\left[\left.X_{t+s}\in A\space\right|X_s\right]=\mathrm E_x\left[Y\left.\space\right|X_s\right],\qquad \text{where}\quad Y=\mathrm{P}_x\left[\left.X_{t+s}\in A\space\right|\mathcal{F}_s\right].$$ Item (3) of Definition 17.3 asserts that $Y$ is $\sigma(X_s)$-measurable, ...

3

The transition kernel $K_t$ is defined by some measurability conditions and by the fact that, for every measurable Borel set $A$ and every (bounded) measurable function $u$, $$\mathrm E(u(X_t):X_{t+1}\in A)=\mathrm E(u(X_t)K_t(X_t,A)).$$ Hence, each $K_t(\cdot,A)$ is defined only up to sets of measure zero for the distribution of $X_t$, in the following ...

3

This uses two conventions at once: For every integer $t\geqslant0$, $P_t$ is the transition matrix of $t$ steps of the Markov chain with transition matrix $P$. Hence $P_0$ is the identity and $P_{t+1}=P_tP=PP_t$ for every $t\geqslant0$. For every transition matrix $P$ and every $B$ in $\Sigma$, $P(x,B)=\sum\limits_{y\in B}P(x,y)$. In the discrete ...

3

It has been a while since I've done anything with Markov chains, so I apologize for any poor notation in advance. Notice that $\{X_n=x_n, X_k=x_k\}$ is $\sigma$-measurable from $\{X_n=x_n, \ldots,\ X_1=x_1 \},$ for $k< n$. Really the Markov property for finite state spaces is equivalent to $P(X_{n+1}=x_{n+1}\,|\, \sigma\{X_n,\, \ldots,\, X_1 \} ... 2 The reason number 2 is wrong is that each number is predicated upon being able to either go to or come from 5 in each state, which does not uniformly affect each of the states. The degree to which each state's steady state varies depends upon the column corresponding to 5 in the transition matrix, which is not the same for each. 2 You are using some non-usual formulas for the kernels of the Markov process. I think, it better to start with the following: $$\mathsf P_x(X_{t+h}\in B|\mathscr F_t) = \mathsf P_{X_t}(X_h\in B)=P^h(X_t,B)$$ where the first identity is the definition of the Markov property, and the second one is the definition of the kernel. Clearly, just by writing ... 2 The Wiki article is a bit misleading: If the bag has 5 each of quarters, nickels, and dimes, then after 15 draws the bag is empty, so there can be no X_16. Even though the example mentioned in the article uses the countable time space of all positive integers {1,2,3...}, the discussion indicates the coins are not being replaced after each draw. So they will ... 2 A sufficient condition is that the process$X$is measurable, that is, the map$(\omega,t)\to X(\omega,t)$should be measurable on the product space$(\Omega\times I,{\cal F}\times {\cal I})$to$(E,{\cal E})$. Then if$\tau:(\Omega,{\cal F})\to(I,{\cal I})$is a (measurable) random time, the composition $$\begin{array}{ccccc} \omega &\to& ... 2 Denote the probability whose limit for N\to\infty you're looking for by P_N. Then P_N satisfies the recurrence relation$$ P_N=\frac14\left(P_{N-2}+P_{N-1}+P_{N+1}+P_{N+2}\right)\;, $$which leads to the characteristic equation$$ \lambda^4+\lambda^3-4\lambda^2+\lambda+1=0\;. $$The double root at \lambda=1 is readily guessed, and the resulting ... 2 For your first question, consider the function z = \max(x_1,x_2). Fix x_2=1 and let x_1 range over [0,2]. Then it is easy to see that z is non-linear, for it is constant for x_1 \in [0,1], but increasing for x_2 \in [1,2]. As to getting around this problem, when you introduce the dummy variable u with the conditions u \geq \mathbf p_j^T ... 2 The answer is: not necessarily. We begin with some partial positive results. Recall that for every pure birth-process (X_t)_{t\geqslant0} with positive rates (\lambda(k))_{k\geqslant0} and for every suitable function u,$$ \frac{\mathrm d}{\mathrm dt}\mathrm E(u(X_t))=\mathrm E((u(X_t+1)-u(X_t))\cdot\lambda(X_t)). $$In particular, the expectation ... 2 For every r let \theta_r denote the time-shift by r. Then,$$ [B_t\in\Gamma,T_0\gt t]=[T_0\gt s,B_s+(B_t-B_s)\in\Gamma,T_0\circ\theta_s\gt t-s], $$where [T_0\gt s] and B_s are \mathcal F_s-measurable and B_t-B_s is independent of \mathcal F_s and distributed like B_{t-s}. Hence,$$ \mathbb P^x(B_t\in\Gamma,T_0\gt t\mid\mathcal F_s)=\mathbf ... 2 We have$n=|S|=5$: the states are the individual pages, so for 3 engines returning 5 possibly duplicate pages, you could have anywhere from 5 to 15 states. Define the relation between pages$j\prec i$if and only if$j$is ranked better than$i$in a majority of engines. This relation is a partial order, and lower elements are "better". The intuitive goal ... 2 If by "stationary" you mean that the transition probabilities$P(X_{n+1} = y \mid X_n = x)$are independent of$n$(also called "time homogeneous"), then the answer is certainly no, if the chain can have an infinite number of states. Consider a chain on the integers where at each step you move one unit to the right with probability 1 (i.e.$P(X_{n+1} = x+1 ...

2

The notation can be a bit cumbersome because of the nested integrals, but this solution relies only on very basic properties of integration and is direct (no induction). Consider the difference $a(x_0,F)=\mathsf P'_{x_0}(F)-\mathsf P_{x_0}(F)$. By uniqueness of measure it follows from the definition of $\mathsf P_{x_0}$ that: \begin{align} a(x_0,F) ...

Only top voted, non community-wiki answers of a minimum length are eligible