A stochastic, or random, process describes the correlation or evolution of random events. It is used to model stock market fluctuations and electronic/audio-visual/biological signals. Among the most well-known stochastic processes are random walks and Brownian motion.

learn more… | top users | synonyms

7
votes
3answers
887 views

Criteria for being a true martingale

Could you kindly list here all the criteria you know which guarantee that a continuous local martingale is in fact a true martingale? Which of these are valid for a general local martingale (non ...
0
votes
1answer
296 views

Optimal stopping for Markov Chain

If the problem of optimal stopping for finite state discrete time Markov Chains is solved on the infinite horizon explicitly? Edited: This means if for a given MC $X(n)$ with a state space ...
1
vote
1answer
257 views

Mix of Poisson processes

Suppose we had a machine where two types of jobs arrive. Jobs of type 1 arrive according to a Poisson process with a rate of $\lambda_1 = 45$ jobs per hour and need an exponential service time with a ...
4
votes
2answers
144 views

When is the expected number of events per period the inverse of the expected duration between events?

I'm trying to model the time between successive events in a sequence of events. Let $T_i$ ($i=1,2,\ldots$) be the time between event $i$ and $i+1$. Assume that the $T_i$ are independent and ...
9
votes
4answers
1k views

Showing that Brownian motion is bounded with non-zero probability

How do you show, that for every bound $\epsilon$, there is a non-zero probability that the motion is bounded on a finite interval. i.e. $$\mathbb{P} (\sup_{t\in[0,1]} |B(t)| < \epsilon) > 0$$ I ...
3
votes
2answers
237 views

What is the average rotation angle needed to change the color of a sphere?

A sphere is painted in black and white. We are looking in the direction of the center of the sphere and see, in the direction of our vision, a point with a given color. When the sphere is rotated, at ...
2
votes
1answer
230 views

Solution to an SDE

This question isn't particularly interesting, but it is frustrating me. Is there a known solution to the stochastic differential equation $$dX_t = (a + bX_t)dt + v X_t dW_t$$ where $W_t$ is standard ...
1
vote
1answer
438 views

How to solve forward equation for a continuous-time Markov chain?

Given the transition rate matrix of a CTMC as $G$, I was wondering how the forward equation $P'(t) = P(t) G, P(0)=I$ is usually solved for the transition matrix $P(t)$? Some book says the ...
3
votes
3answers
360 views

How to evaluate the following stochastic integral?

How to evaluate the following stochastic integral? $$\int_0^t M_{s^-}^2 dM_s$$ where $M_t = N_t - \lambda t$ is a compensated Poisson process. I tried to apply Ito's formula to $M_t^2$ but still ...
1
vote
1answer
90 views

Distribution of service time when server restarts can occur

(This is a differently formulated version of the question. There were no answers, comments or votes on the first version so I thought I'd give it another shot.) Suppose a server processes jobs that ...
1
vote
1answer
119 views

Queueing Theory - Probability that all jobs have been served?

Suppose I have M/M/1 system with $\lambda = 4$ per hour and $\mu = 5$ per hour. How can I find out if all jobs have been served after, say, 8 hours? At first I thought about doing $P(n > 40)$ since ...
0
votes
2answers
623 views

Quadratic variation of continuous local martingales

Dear all, I hope you can help me with the proof of the following result: Fact If $X$ is a continuous local martingale, then $[X]_t < \infty $ a.s. for every $t \geq 0$, where $[X]$ denote the ...
1
vote
1answer
255 views

Continuous a.s. process

In Ross's Stochastic processes: A stochastic process $\{X(t), t \geq 0\}$ is said to be a Brownian motion process if $X(0) = 0$, $\{X(t), t \geq 0\}$ has stationary independent ...
4
votes
2answers
297 views

A probability question

Suppose $X_1, X_2, ...,$ are IID random variables with $P(X_n=1)=p$ and $P(X_n=2)=1-p$. Let $S_n=\sum_{i=1}^n X_i$. I was wondering how to find $P(S_n \neq z, \forall n \in \mathbb{N})$ for some ...
1
vote
1answer
278 views

Convergence of positive martingale

Suppose $(B_t)_{t\geq 0}$ is a Brownian motion and that $S_t = \exp (B_t-\frac{t}{2})$. By the martingale convergence theorem, $S_t\to S_\infty$, some random constant, a.s.. It seems that we should ...
1
vote
3answers
287 views

Markov property with respect to a filtration

Suppose $\{ X_t: t \in \mathbb{R} \}$ is a stochastic process on a probability space $(\Omega, \mathcal{F}, P)$, and it is adapted to a filtration $\{\mathcal{F}_t \}$ on the probability space. $\{ ...
3
votes
3answers
582 views

asymmetric random walk

do asymmetric random walks also return to the origin infinitely?
0
votes
2answers
341 views

Questions about a birth and death process

My question has a queue M/M/1/2, that is, a system with exponential interarrival and service times, one server and having a room only for 2 customers (including the one in service, and another that is ...
1
vote
0answers
649 views

Partial differentiation of vector to find Jacobian (extended Kalman filter)

I am working through some coursework on self-tuning control and part of one of the questions requires the use of the extended Kalman filter for joint parameter and state estimation. For completeness, ...
5
votes
1answer
400 views

Crowded and quiet periods in a $M/M/1$ queue

I'm trying to solve the following exercise (not homework): Consider a $M/M/1$ queue with an arrival rate of 60 customers per hour and a mean service time of 45 seconds. A period during which there ...
1
vote
2answers
170 views

Independence of holding time and next state in continuous-time Markov chain

In a continuous-time Markov chain, I was wondering why the holding time and the next state are independent? Are the independence a conditional one given the current state? Quoted from Ross's ...
2
votes
2answers
874 views

Is the limit of power of a stochastic matrix still a stochastic matrix?

Suppose $A$ is a right stochastic matrix, which is defined as a square matrix each of whose rows consists of nonnegative real numbers, with each row summing to 1. If $\lim_{n \rightarrow \infty} ...
1
vote
1answer
72 views

Are probabilities / expected times of transition between any two states within an irreducible class same?

In a discrete time Markov chain, consider an irreducible/communicating class, Are the probabilities of ever transition between any two states within the class the same? If the class is recurrent, ...
2
votes
0answers
440 views

Questions about stationary and limiting distributions of a discrete-time Markov chain

For a discrete-time Markov chain, Is it right that there are no more than one limiting distribution, i.e., limiting distribution is unique if any? If the chain has more than one recurrence ...
1
vote
1answer
456 views

The infimum of a drifting Brownian motion

Using the reflection principle, it's easily shown that the infimum $\displaystyle \inf_{u \in [0, T]} B_u$ of a non-drifting Brownian motion $B_t$ has the same distribution as $-| B_T |$. The ...
1
vote
1answer
2k views

Relation between independent increments and Markov property

Independent increments and Markov property.do not imply each other. I was wondering if being one makes a process closer to being the other? if there are cases where one implies the other? ...
0
votes
1answer
306 views

Are holding times independent in a continuous-time Markov chain and in a semi-Markov process

I was wondering if the holding times are independent in a continuous-time Markov chain? Similar question in a semi-Markov process? From what I have read, it is not mentioned that the holding times ...
2
votes
1answer
276 views

Markov chain exit time

Consider a reversible markov chain $X_t$ defined on a square lattice, with transition probabilities defined between adjacent vertices. Take a square subset of the lattice and call it $V$. Let $dV$ be ...
0
votes
1answer
175 views

Covariance of Brownian Motion

What is the covariance function for $U(t)$ if $U(t) = e^{-t}B(e^{2t})$ for $t \geq 0$ where $B(t)$ is standard Brownian motion? Any help would be great
2
votes
1answer
554 views

Determining the balance equations for a Poisson Process

I'm trying to do an exercise (not homework) and I fail to understand the solution the reader is giving me. Consider a gas station with one gas pump. Cars arrive at the gas station according to a ...
1
vote
2answers
196 views

Expected sojourn time for a pinned brownian motion

The problem I am trying to solve is the following: What is the expected time the Wiener process $W_{t}$ stays above t-axis for $t\in [0;1]$ if we know that $W_{1}=a$? I suppose that this is a well ...
1
vote
1answer
242 views

Continuous Time Markov Chains

What are some techniques to convert Continuous Time Markov Processes into Discrete time Markov Processes? (for purposes of simulations)
2
votes
2answers
246 views

incremental simulation of GBM

(I asked this question in stackoverflow.com, but I am now thinking my mistake may be mathematical rather than programming). I am simulating geometric brownian motion, using closed-form solution for ...
2
votes
1answer
266 views

Brownian motion

Verify that $E( X(t) X(s) | X(0)=0 ) = min (t, s)$, where $X(t)$ is standard Brownian motion. I don't know where to start. Thanks!
4
votes
1answer
534 views

Splitting a Poisson process according to time-dependent probabilities

Let $X_t$ be a homogeneous Poisson process of rate $\lambda$. Suppose we define functions $p_1(t)$, ..., $p_k(t)$, such that for all $i$ and $t$, $p_i(t)\in [0,1]$ and $\sum\limits_{i=1}^kp_i(t) = 1$. ...
3
votes
1answer
203 views

Poisson arrivals followed by locking

following is my problem: Pulses arrive at a processor according to a Poisson process of rate λ. Suppose each arriving pulse that is processed by the processor locks the processor for a fixed time T, ...
1
vote
1answer
258 views

Limiting distribution of alternating renewal process

Consider an alternating renewal system that can be in one of two states: on or off. Initially it is on and it remains on for a time $Z_1$, it then goes off and remains off for a time $Y_1$, it then ...
2
votes
0answers
91 views

Measurability of a point process or random measure at a measurable subset

Suppose $\xi$ is a point process on $(S, B(S))$, where $S$ be locally compact second countable Hausdorff space equipped with its Borel σ-algebra $B(S)$. I was wondering if $\xi(A), \forall ...
2
votes
1answer
444 views

Confusions regarding the concept of a stopping time for a martingale

I am studying martingales and I have a few conceptual questions regarding why we need stopping times. My book (Probability and Computing by Mitzenmacher and Upfal) defines a martingale as follows: A ...
6
votes
2answers
505 views

Expected travel time for regularly departing trains

I'm going to ask a very simple practical question, but I believe it has some interesting mathematical properties. The simple variant is: trains depart every $x$ minutes and take $y$ minutes to arrive ...
1
vote
1answer
227 views

Question for understanding definition of point process

I am trying to understand the definition of point process when reading its Wikipedia article: Let $S$ be locally compact second countable Hausdorff space equipped with its Borel σ-algebra ...
4
votes
1answer
883 views

Stochastic integral with a Poisson process

I have a Poisson process $X_t$ for $t\ge0$. How I can find a process $b_t$ such that $$\exp ({\alpha X_t})=1+\int_0^t b_{s^{-}}dX_s$$ where $\alpha\in\mathbb{R}$ and what would be the expectation of ...
0
votes
1answer
106 views

Backward representation of the general Markov process

One usually deals with a discrete time Markov process in the following form: given a state space $E$ the Markov process is defined by transition kernel $T(B|x)$ such that $$ \mathsf{P}(X_1\in B|X_0 = ...
3
votes
1answer
100 views

optimized upper bound; stochastic

Let $I_t=\int_0^t f_tdB_t,$ where $(f_t,t\ge 0)$ is a bounded process, $|f_t|\leq M$ almost surely for all $t \ge 0.$ Show that $$\mathcal{P}\left[\sup_{0\leq t\leq T}|I_t|>\lambda\right]\leq ...
1
vote
1answer
80 views

1-dimensional diffusion process

Let $(X_t, t ≥ 0)$ be a 1-dimensional diffusion process with generator $Af(x) =\frac{1}{2}a(x)f''(x)+b(x)f'(x), \mathcal{D}(A)=C^2({\mathbb{R}})$ where $b$ and $a=\sigma^2$ are continuous ...
1
vote
2answers
2k views

First exit time for Brownian motion without drift

I am dealing with the simulation of particles exhibiting Brownian motion without drift, currently by updating the position in given time steps $\Delta t$ by random displacement in each direction drawn ...
2
votes
2answers
570 views

Characterization of two-step 2x2 stochastic matrices

Show that: A 2 x 2 stochastic matrix is two-step transition matrix of a Markov chain if and only if the sum of its principal diagonal terms is greater than or equal to $1$.
1
vote
2answers
160 views

modified stochastic process

Is there any study of stochastic processes where the probability matrix (for a finite state process) is time dependent? For example, probability I go from school to home is higher at night as ...
0
votes
0answers
59 views

Randomized methods

Randomized methods are often used in the probability theory as a kind fo numerical methods to obtain some results which cannot be easily (or even hardly) obtained analytically. One of the most famous ...
10
votes
2answers
259 views

A sequence of order statistics from an iid sequence

Given a sequence of iid random variables $X_i$ (without loss of generality from $U(0,1)$), an integer $k \ge 1$ and some $p \in (0,1)$, construct the sequence of random vectors $Z^{(j)}$, $j=0,1,...$ ...