2
votes
1answer
53 views

Filling of a tank - recurrence relation

Suppose a tank has a maximum limit of 100 units. Each day 2,1 and 0 units are added to the water level with probability p,r and q. Any excess water would overflow and if it reaches the minimum level ...
1
vote
1answer
54 views

Need help with a basic exercise about Markov chains

Suppose $\left\{ X_{n}\right\} _{n=1}^{\infty}$ is a Markov Chain taking real values. Are the following Markov Chains? $$Y_{n}=\sum_{i=1}^{n}X_{i} , Z_{n}=\left(X_{n},X_{n-1}\right)$$ Edit1 I ...
0
votes
1answer
18 views

Random process with Cauchy distribution

The problem is as follows. Let $X(t)$ be a stochastic process such that $X(t) = V + 2t, t \ge 0$, and $V$ has the Cauchy distribution $x_0 = 0, \gamma = 1$. Find the probability that $X(t) = 0$ for ...
1
vote
0answers
29 views

A question on recurrent events

In a sequence of Bernoulli trials let E occur when the accumulated number of successes equal to $c$ times the number of failures where $c$ is a positive integer. I need to show that E is persistent if ...
1
vote
1answer
25 views

Generating two $-1$ correlated Poisson random variables with parameter $5$

Is it possible to generate two random variables $X$ and $Y$ that are both $Poisson(5)$ with $Corr(X,Y)=-1$? Why? I was thinking about generating $3$ independent Poisson random variables $Z_1,Z_2, and ...
2
votes
0answers
40 views

A right-inverse of Brownian motion local time at zero has stationary independent increments

Let $L_0^t$ be the local time for a standard Brownian motion at $0$ and define $$X_t=\sup\{s\ge0:L_0^s\le t\}, t\ge0. $$ I would like to show that $(X_t)$ has stationary independent increments. That ...
0
votes
1answer
47 views

Birth and Death Process Questions

Consider a birth and death process with the birth rate $\lambda_m = \lambda (m\ge 0)$ and death rate $\mu_m = m \mu (m \ge 1)$. A. How would I derive the stationary distribution? Only information I ...
0
votes
0answers
16 views

Markov Chain Steady State Probabilities

This is a hw question that I'm not sure how figure out. I have this markov chain - I'm supposed to figure out if steady state probabilities exist if I start in state 1. I know that I should ...
0
votes
1answer
40 views

Gaussian vectors and covariance matrix.

The following is a part of a question I was given in stochastic processes course. It goes like this - I am given a series of gaussian iid random variables $\{V_i\}_{i=1}^N$ , the variable $X_0 \sim ...
0
votes
0answers
22 views

Distribution and Laplace transform

I'm having trouble understanding this problem from Resnick's Adventures in Stochastic Processes: The problem says: Suppose $F$ is a distribution of a positive random variable and $p_k \geq 0, ...
1
vote
1answer
43 views

If $dX_t=b_tdt+\sigma_tdW_t=\tilde{b}_tdt+\tilde{\sigma}_tdW_t$ then $b_t=\tilde{b}_t$ and $\sigma_t=\tilde{\sigma}_t$ a.s

Let $X_t$ be an Ito's process where $dX_t=b_tdt+\sigma_tdW_t=\tilde{b}_tdt+\tilde{\sigma}_tdW_t$. Prove $b_t=\tilde{b}_t$ and $\sigma_t=\tilde{\sigma}_t$ a.s Here my solution for ...
0
votes
2answers
33 views

If $u(z),$ where $Z_t=W^1_t-iW^2_t$, is a complex anlyt. fx, show $du(Z_t)=u'(Z_t)dZ_t$

If $u(z)$ is a complex analytical function, where $Z_t=W^1_t-iW^2_t$ is a complex Wiener process, show $du(Z_t)=u'(Z_t)dZ_t$.
0
votes
1answer
43 views

Quadratic Variation for $X_t= \int b_s ds$ where $b_s$ is an $F_t$ adapted process.

Let $b_S$ be an $F_t$ adapted process, Borel measurable in $t$ st $\int |b_s|^2ds < \infty$ (a.s). Setting $X_t=\int^t_0 b_sds$ and partitioning the interval $[0,t]$ i.e. $0=t^n_0<t^n_1... $ ...
2
votes
1answer
49 views

Probability distribution of Poisson process

Let $X_t$ and $Y_t$ be two independent Poisson process with rate parameter $\lambda_1$ and $\lambda_2$, respectively, measuring the number of customers arriving in stores $1$ and $2$, respectively. ...
0
votes
0answers
127 views

Expectation of a Poisson Process

Cars pass a certain street location according to a Poisson Process with rate $\lambda$. An old lady and her trusty boyscout want to cross the street at this location. They wait until they can ensure ...
3
votes
0answers
44 views

Conditional expectation and coupled set of ODEs

How to find a coupled set of ODEs and initial conditions for the deterministic functions $a$ and $b$ such that $$\mathbb{E}\left[e^{-\int_{t}^{T} W^2(u)du} | \mathcal{F(t)}\right] = e^{-a(T-t) - ...
0
votes
1answer
52 views

Proving a chain is aperiodic, and finding a stationary distribution.

We have an irreducible Markov chain with a not necessarily finite state space. It has a transition matrix $P$ such that $P^2=P$. Prove (1) the chain is aperiodic, and (2) prove $p_{ij}=p_{jj}$ ...
0
votes
1answer
23 views

Question involving an invariant measure on a Markov chain

Suppose $\mu$ is an invariant measure for a Markov chain with state space $S$ with $\mu(i)p_{ij}=\mu(j)p_{ji}$ $\forall i,j \in S$. Describe a Markov chain with this property. Also, show that $\mu$ is ...
1
vote
2answers
61 views

Poisson Process and Conditional Probability

Let $X= (X(t); t\ge0)$ be a poisson process with the intensity ($\lambda$ per hour) A) Find the conditional probability of having $m$ events in the first $t$ hours, given that there were $n$ events ...
1
vote
1answer
90 views

Markov Chain: prove that state is positive recurrent by calculating expected # of transitions to return to this state

Given the transitional probabilities below (states: 0,1,2,3), I need to prove that state 3 is positive recurrent by calculating expected # of transitions to return to this state $$P = ...
5
votes
2answers
105 views

Estimating the maximum of a Brownian motion over the unit interval

Let $\left(B_t\right)_{t \in \left[0,\infty\right)}$ be a standard Brownian motion over the probability space $\left(\Omega, \mathcal{A}, P\right)$. For each $x \in \left(0, \infty\right)$, give an ...
0
votes
0answers
60 views

Inventory Control using Dynamic Programming

I am trying to solve a traditional inventory control stochastic dynamic programming problem where \begin{align} x_{k+1} &= x_k + u_k - w_k\\ w_{N-1} &= \begin{cases} 0 &\text{w.p. } ...
0
votes
2answers
54 views

Show $L$ is not a stopping time

Let $L = \sup\{ n : n \le 10; A_n \in B \}$, $B \in \mathcal B$, $\sup\{\emptyset \}=0$. $(A_n)_{n \ge1}$ is a process adapted by a natural filtration $\{\mathcal F_n\}.$ Show that $L$ is NOT a ...
0
votes
1answer
36 views

Markov Chain Conditional Probability

A Markov chain has the transition probability matrix as follows. $$To$$ $$ From \begin{matrix} STATES& 0 & 1 & 2 \\ 0 & 0.6 & 0.3 & 0.1 \\ 1 ...
1
vote
1answer
63 views

Expectation of stochastic differential equation

I have solved the nonlinear stochastic equation $dX_t=\frac{1}{2}a(a-1)X_t^{1-2/a}dt+aX_t^{1-1/a}dW_{t}$, by reducing it to a linear one (change of variables $Y_{t}=X_{t}^{1/a}$ and applying Ito ...
1
vote
0answers
37 views

Markov Chain depicting unruly customer behavior

A store has 2 bins of balls. 1 bin is red, and contains 3 red balls. The other bin is gray and contains 2 gray balls. Every minute, on the minute, exactly one customer comes by the bins, picks up ...
0
votes
2answers
91 views

CDF and PDF of transformed variables from a uniform distribution

A random variable U follows the uniform distribution of (-1,1). Find the cumulative distribution functions and density for the transformed variables listed below. a. $X = |U|$ b. $Y = -ln(|U|)$ c. ...
0
votes
3answers
48 views

Unif~(-1,1) and finding min and max for two independent random variables

Two independent random variables, $X$ and $Y$, are uniformly distributed on the unit interval (-1,1). Determine the density for $U=min(X,Y)$ and for $W=max(X,Y)$ Find the expectation for each ...
1
vote
1answer
81 views

How to get Expectation and Variance of Geometric Distributions

A random variable N follows a geometric distribution with the "Success Rate" = $\alpha$, so that $P [N=k] = \alpha (1-\alpha)^{k-1},$ for $k$ = 1,2,... Random variables $(Y_1 , Y_2...)$ are ...
0
votes
0answers
36 views

Expectation, Variance, and Value of a Constant

I think I have everything right, I would like someone to check my work. A random variable X has the following density function (Beta-Distribution) (Density) = f(x) = Cx^2 (1-x)^2 (If 0 < x < 1 ) ...
1
vote
1answer
55 views

Expectation and covariance of a gamma distribution.

Assume that the conditional distribution of $U$, given $L$ is uniform over the interval $[0,L]$ and $L$ itself has the gamma-distribution with the density described below. \begin{equation} ...
1
vote
1answer
47 views

How to express the following probability expectation?

A total of n bar magnets are bent slightly such that they can form a circle. There is 1/2 chance for each bar magnet to be in any orientation. Adjacent magents with opposit poles facing each other ...
0
votes
0answers
36 views

Discrete Time Markov Chain - Store Inventory

Let $D_n$ be the demand for an item at a store on day $n$. Suppose that $D_n$ is a sequence of independent and identically distributed (i.i.d.) random variables with probability mass function: $p_k = ...
0
votes
0answers
46 views

Discrete Time Markov Chain - Bank customers

Suppose $N_n$, the number of customers arriving at a bank during day $n$, is distributed Binomial$(p, m)$. Consider the simple situation where each customer either withdraws £1 or deposits £1 with ...
1
vote
1answer
56 views

Finding the probability of occurrence of one event before an another

In a home work problem $E$ and $F$ are mutually exclusive events in the sample space of an experiment. The experiment is repeated until either event $E$ or event $F$ occurs. Show that the probability ...
0
votes
1answer
48 views

Poisson Process (Easy)

I'm stuck at the following question: Customers with items to repair arrive at a repair facility according to a Poisson process with rate λ. The repair time of an item has a uniform distribution on ...
2
votes
0answers
99 views

Almost sure non differentiability of Brownian Motion

Problem: Let $t>0$, show that the standard Brownian motion is almost surely not differentiable a $t$ Now, through a Borel Cantelli argument I proved that, almost surely $$\limsup_{\epsilon ...
1
vote
0answers
32 views

period of product markov chain

Consider $Z_n := (X_n,Y_n)$ where $(X_n)_{n\in \mathbb{N}}$ and $(Y_n)_{n\in \mathbb{N}}$ are irreducible markov chains with periods $\lambda$ and $\mu$. We know that $(Z_n)_{n\in \mathbb{N}}$ is a ...
5
votes
2answers
80 views

Variance of drawing coins from a bag.

First off, disclaimer, this was a homework question, albeit one that I've already turned in. I was given the problem ...
2
votes
0answers
149 views

Local martingale is locally uniformly integrable martingale?

Is a local martingale locally uniformly integrable martingale ? Here I define a local martingale to be the process with a localizing sequence $\tau_n$ such that the stopped process is martingale. ...
0
votes
0answers
51 views

simplified iid Galton-Watson-process — expectation and variance of population size

Let $Y_{ni}$ be iid and take on values in $\{0,1,2,\ldots\}$. Set $Z_0=1$ and define $Z_n:=\sum_{i=1}^{Z_{n-1}}Y_{ni}$ where by convention the sum is zero if $Z_{n-1}=0$. Let $E(Y_{11}) = \mu$ and ...
0
votes
0answers
43 views

Is this two dimensional Markov chain correct for this queueing system?

The problem that I have two single server station with no queuing space a customer goes to station 1 if it is available else it goes to station 2 if it is available or it will be lost output from ...
0
votes
0answers
27 views

Analyze the packet loss rate

Suppose that node A send $m(i)$ ($m(i)$ obeys i.i.d. poisson distribution with parameter $\lambda$) packets to node B at time slot $i$, and B can handle $C$ packets at each time slot. There is a ...
2
votes
1answer
113 views

Stochastic dynamic programming

I am making some homework exercises at the moment and I was wondering if what I did in the following exercise was correct. PROBLEM Solve $E(\sum_{k=0}^{N-1}(1-u_k)X_k + X_N) \rightarrow \max$, ...
0
votes
1answer
48 views

Compute $d(\log(S_t))$ using Ito's Formula

We are given the following: d$S_t$ = $\sin(S_t)t^2dt + e^{\sqrt{S_t}-t}dB_t$ And are asked to compute several different things, one of which is $d \log(S_t).$ If I'm understanding Ito's formula ...
1
vote
0answers
39 views

is this solution correct about joint distribution?

The question is if $x,y,z$ are independent $x\sim\exp(\lambda), y\sim\exp(\mu), z\sim\exp(\gamma)$ and define $u=\min(x,y), v=\min(y,z)$ what is the probability $p(U>u,V>v)$. Consider the cases ...
0
votes
1answer
41 views

Branching process question.

Suppose we have a branching process, where at each time $n$, each individual produces offspring independently with the distribution $\{p_k\}$ and then dies with probability $0 < q < 1$. For ...
0
votes
0answers
43 views

decreasing capacity of channel

I have a question regarding the capacity of a channel Consider a channel given by the transition probabilities $p(y|x)$ with capacity $C$. Now a friendly statistician offers to preprocess the output ...
0
votes
1answer
80 views

Finding the expected value of a stochastic process?

Here is the question: Let X(t)=At+B, where A and B are independent Normal distributions N(0,$\sigma^2$). Find the expected value of the following Random Variables: (A) $Y_1 = max_{0 \leq t ...
0
votes
0answers
52 views

Given an innovations sequence, calculate transformation matrix, A

Let $Y_1,Y_2,Y_3,X)^T$ be a zero mean random vector with correlation matrix, $$ \begin{pmatrix} 2 & 1 & 1 & 2 \\ 1 & 2 & 1 & 2 \\ 1 & 1 & 2 & 2 \\ 2 & 2 & 2 ...