A stochastic, or random, process describes the correlation or evolution of random events. It is used to model stock market fluctuations and electronic/audio-visual/biological signals. Among the most well-known stochastic processes are random walks and Brownian motion.

learn more… | top users | synonyms

42
votes
7answers
11k views

Proving that 1- and 2-d simple symmetric random walks return to the origin with probability 1

How does one prove that a simple (steps of length $1$ in directions parallel to the axes) symmetric (each possible direction is equally likely) random walk in $1$ or $2$ dimensions returns to the ...
3
votes
1answer
119 views

When Superposition of Two Renewal Processes is another Renewal Process?

When superposition of two renewal processes is another renewal process? If you merge (superpose) two Poisson processes with parameters $\lambda_1$ and $\lambda_2$, the outcome is another Poisson ...
11
votes
1answer
7k views

Distribution of hitting time of line by Brownian motion

I came across the following question: Let $T_{a,b}$ denote the first hitting time of the line $a + bs$ by a standard Brownian motion, where $a > 0$ and $−\infty < b < \infty$ and let $T_a ...
5
votes
3answers
995 views

Wiener Process $dB^2=dt$

Why is $dB^2=dt$? Every online source I've come across lists this as an exercise or just states it, but why isn't this ever explicitly proved? I know that $dB=\sqrt{dt}Z$, but I don't know what ...
15
votes
2answers
2k views

Random walk on $n$-cycle

For a graph $G$, let $W$ be the (random) vertex occupied at the first time the random walk has visited every vertex. That is, $W$ is the last new vertex to be visited by the random walk. Prove the ...
5
votes
2answers
2k views

Time until a consecutive sequence of ones in a random bit sequence

This a reformulation of a practical problem I encountered. Say we have an infinite sequence of random, i.i.d bits. For each bit $X_i$, $P(X_i=1)=p$. What is the expected time until we get a sequence ...
7
votes
2answers
4k views

Equilibrium distributions of Markov Chains

I often get confused about when a Markov chain has an equilibrium distribution; when this equilibrium distribution is unique; which starting states converge to the equilibrium distribution; and ...
3
votes
1answer
1k views

Angle bracket and sharp bracket for discontinuous processes

The question is quite simple actually. I am trying to understand the differences between the angle bracket $\left<X,Y\right>$ of two processes with jumps $X,Y$, and the sharp bracket of $[X,Y]$. ...
3
votes
1answer
3k views

Relation between independent increments and Markov property

Independent increments and Markov property.do not imply each other. I was wondering if being one makes a process closer to being the other? if there are cases where one implies the other? ...
4
votes
2answers
490 views

Using Recursion to Solve Coupon Collector

I read a brilliant answer by Mike Spivey on one of the questions and I was wondering how I could use it to solve a coupon collectors problem. The problem is : There are coupons labelled 1,2,3...,10 ...
1
vote
1answer
262 views

Markov processes driven by the noise

Let $\xi_n\in \Xi$ be a sequence of iid random variables with $n \in\mathbb N\cup\{0\}$, which we call a noise process. Construct a process $$ Z_{n+1} = f(Z_n,\xi_n)\quad(\star) $$ with $Z_0\in E$ ...
10
votes
1answer
7k views

equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution

I was wondering if equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution mean the same, or there are differences between? I learn them in the context ...
0
votes
1answer
128 views

Prove $\mathbb{P}(\sup_{t \geq 0} M_t > x \mid \mathcal{F}_0)= 1 \wedge \frac{M_0}{x}$ for a martingale $(M_t)_{t \geq 0}$

Let $M$ be a positive, continuous martingale that converges a.s. to zero as $t$ tends to infinity. I now want to prove that for every $x>0$ $$ P\left( \sup_{t \geq 0 } M_t > x \mid \mathcal{F}_0 ...
19
votes
1answer
4k views

Interpretation of sigma algebra

My question is how to interpret sigma algebra, especially in the context of probability theory (stochastic processes included). I would like to know if there is some clear and general way to interpret ...
12
votes
3answers
2k views

Why is stopping time defined as a random variable?

I've been given a crash course in stochastic processes and martingales for the purposes of a semester project on them. The guy I'm working with has been, I feel, a little vague in the definition of ...
18
votes
2answers
2k views

What is meant by a continuous-time white noise process?

What is meant by a continuous-time white noise process? In a discussion following a question a few months ago, I stated that as an engineer, I am used to thinking of a continuous-time ...
13
votes
3answers
431 views

Limit of a Wiener integral

How to show that $$ \lim _{\alpha \rightarrow \infty } \sup_{t \in \left [0,T \right]} \left | e^{-\alpha t} \int _ 0 ^t e^{\alpha s} ~ dB_s \right | =0, \ \ \text{a.e.} $$ where $\left (B_s ...
2
votes
2answers
4k views

First exit time for Brownian motion without drift

I am dealing with the simulation of particles exhibiting Brownian motion without drift, currently by updating the position in given time steps $\Delta t$ by random displacement in each direction drawn ...
7
votes
3answers
883 views

Difference between Modification and Indistinguishable

Would someone be able to offer a layman's explanation of what is means when two stochastic processes are a Modification of each other and when they are Indistinguishable? My Stochastic Analysis notes ...
5
votes
2answers
590 views

Logarithm of a Markov Matrix

Start with a Markov matrix $\mathbf{M}$, whose elements are all between $0 \le \mathbf{M}_{ij} \le 1$ and each row sums to one. There is a natural connection with this matrix and the rate matrix ...
1
vote
1answer
1k views

Scalar product of Gaussian process

Assume that $n(t)$ is a White Gaussian Noise (WGN) process with $E[n(t)]=0$, $E[n(t)^2]=\sigma^2$ and $x(t)$ a deterministic function defined in $[0,T]$. How can I compute from first principles the ...
3
votes
1answer
447 views

Random walk $< 0$

Suppose ${X_t}$ is a random walk with mean zero. (either discrete or continuous time) Fix a time $T$. What is: $P[X_t < 0 \text{ for all } t \leq T]$? In words, what's the probability the random ...
2
votes
1answer
1k views

When is a Markov process independent-increment?

An independent-increment stochastic process must be Markov. I am now wondering about the reverse case. Why do some Markov processes fail to be independent-increment? What are some examples of Markov ...
1
vote
1answer
297 views

When random walk is upper unbounded

Consider a random walk $S_n = a_1+\dots+a_n$ where $a_n$ are iid random variables with $Ea_1 = a$ and $E|a_1|<\infty$. I am interested in the case when $\sup\limits_n S_n>M$ for all $M$ a.s. ...
0
votes
1answer
61 views

Asymmetric Random Walk / Prove that $T:= \inf\{n: X_n = b\}$ is a $\{\mathscr F_n\}_{n \in \mathbb N}$-stopping time

Given random variables $Y_1, Y_2, ... \stackrel{iid}{\sim} P(Y_i = 1) = p = 1 - q = 1 - P(Y_i = -1)$ where $p > q$ in a filtered probability space $(\Omega, \mathscr F, \{\mathscr F_n\}_{n \in ...
0
votes
1answer
212 views

Prove Z is a martingale by defining it is a product of random variables

Given a filtered probability space $(\Omega, \mathscr{F}, \{\mathscr{F_n}\}, \mathbb{P})$ where $\mathscr{F_n} = \mathscr{F_n}^{Z} \doteq \sigma(Z_0, Z_1, \ldots, Z_n)$, show that $Z = (Z_n)_{n \geq ...
63
votes
4answers
5k views

Why did my friend lose all his money?

Not sure if this is a question for math.se or stats.se, but here we go: Our MUD (Multi-User-Dungeon, a sort of textbased world of warcraft) has a casino where players can play a simple roulette. My ...
20
votes
2answers
2k views

What is the importance of the infinitesimal generator of Brownian motion?

I have read that the infinitesimal generator of Brownian motion is $\frac{1}{2}\small\triangle$. Unfortunately, I have no background in semigroup theory, and the expositions of semigroup theory I have ...
7
votes
2answers
3k views

Prove the time inversion formula is brownian motion

Let $B=(B_t)_{t\geq 0}$ be a brownian motion. Show the time inversion formula $\hat{B}=(B_t)_t\geq0$ is a brownian motion, where for $t \geq 0$ we set $\hat{B}=0$ for $t=0$ and $\hat{B}=tB_{1/t}$ for ...
5
votes
2answers
6k views

How to characterize recurrent and transient states of Markov chain

According to Wikipedia with a little rephrasing: A state $i$ is transient if and only if $P(T_i < \infty) <1$, recurrent if and only if $P(T_i < \infty) =1$, where $T_i$ is the ...
7
votes
4answers
5k views

Sum and product of Martingale processes

Given two Martingale processes $(X_t)$ and $(Y_t)$, are their sum $(X_t+Y_t)$ and their product $(X_t \times Y_t)$ also Martingale? If not, will the two $(X_t)$ and $(Y_t)$ being independent grant ...
4
votes
1answer
287 views

“Continuity” of stochastic integral wrt Brownian motion

I'd like to prove a nice property of a stochastic integral with respect to Brownian motion. Let $(H_t)_{t\geq0}$ be a progressive and bounded process that is continuous at $0$ and $B$ a standard ...
9
votes
2answers
878 views

Joint moments of Brownian motion

My approach to this SE question uses the following joint moments of Brownian motion. For $n=1,2$ they are obvious and well-known, the others are not terribly hard to work out. Is there a reference ...
8
votes
2answers
2k views

Prokhorov metric vs. total variation norm

Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ ...
7
votes
2answers
1k views

Dominated convergence problems with Wald's identity for the Brownian Motion

In the course of proving Wald's second identity $E(B^2_T)=E(T)$, where $(B_t)_{t\geq0}$ is the Brownian motion and $T$ is a stopping time with $E(T)<\infty$, I got stuck with the following problem. ...
6
votes
1answer
453 views

Right continuous version of a martingale

This is an exercise in chapter 2 of the book "Continuous Martingales and Brownian Motion" by Revuz and Yor: Consider the probability space $([0,1], \mathcal{B}([0,1]), dx)$, where $dx$ denotes ...
7
votes
1answer
719 views

Applications of algebra and/or topology to stochastic (or Markov) processes

Some time back I was reading a PDF about algebra or topology (or algebraic topology, I forget which) and found an extremely enlightening section about an application to stochastic processes. ...
4
votes
0answers
244 views

6-digit password - a special decoding method

Consider the situation of decoding a 6-digit password that consists of the symbols A to Z and 0 to 9, where all possible combinations are tried randomly and uniformly. Consider the ...
2
votes
1answer
1k views

Is continuous L2 bounded local martingale a true martingale?

I can prove it briefly, but I found a "counter" example. (There must be a mistake in the following words...) I can prove: X is a continuous local martingale, with $X_0=0$ a.s, then X is $L_2$ bounded ...
6
votes
1answer
358 views

Feller continuity of the stochastic kernel

Given a metric space $X$ with a Borel sigma-algebra, the stochastic kernel $K(x,B)$ is such that $x\mapsto K(x,B)$ is a measurable function and a $B\mapsto K(x,B)$ is a probability measure on $X$ for ...
5
votes
2answers
689 views

Is the condition “sample paths are continuous” an appropriate part of the “characterization” of the Wiener process?

Wikipedia has separate articles on "Brownian motion" and "Wiener process" (http://en.wikipedia.org/wiki/Brownian_motion and http://en.wikipedia.org/wiki/Wiener_process ). I am not an expert, but that ...
4
votes
1answer
105 views

Is the distribution of an Ito diffusion at time t absolutely continuous wrt Lebesgue measure?

Suppose we have an sde of the form: \begin{eqnarray} dX_t=b(X_t)dX_t + \sigma (X_t)dB_t \end{eqnarray} where $b$ and $\sigma$ are Lipschitz. Then we have existence and uniqueness of the solution ...
1
vote
2answers
159 views

How to model multi-step cell differentiation

Can I better explain cell lineages using PDEs or stochastic?
17
votes
2answers
5k views

Nice references on Markov chains/processes?

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes ...
14
votes
4answers
2k views

Showing that Brownian motion is bounded with non-zero probability

How do you show, that for every bound $\epsilon$, there is a non-zero probability that the motion is bounded on a finite interval. i.e. $$\mathbb{P} (\sup_{t\in[0,1]} |B(t)| < \epsilon) > 0$$ I ...
10
votes
3answers
2k views

Criteria for being a true martingale

Could you kindly list here all the criteria you know which guarantee that a continuous local martingale is in fact a true martingale? Which of these are valid for a general local martingale (non ...
8
votes
1answer
202 views

When does $\sum_{i=1}^{\infty} X_i$ exist for random sequences $\{X_i\}_{i=1}^{\infty}$?

Suppose $\{X_1, X_2, X_3, \ldots\}$ is an infinite sequence of random variables such that $E[X_i]=0$ for all $i$, and $E[X_iX_j]=0$ whenever $i \neq j$. Further suppose the variances $\sigma_i^2 = ...
6
votes
3answers
7k views

What is the difference between all types of Markov Chains?

I have been looking for some good material covering Markov Chains but everything seems so difficult to me... After reading about the subject, I figured out that there is basically three kinds of ...
14
votes
3answers
3k views

The Laplace transform of the first hitting time of Brownian motion

Let $B_t$ be the standard Brownian motion process, $a > 0$, and let $H_a = \inf \{ t : B_t > a \}$ be a stopping time. I want to show that the Laplace transform of $H_a$ is ...
9
votes
1answer
2k views

Recommendation on stochastic process books

I was wondering if someone could recommend good books on stochastic processes with measure theory treatment with not much or no measure theory treatment for each, it would be nice to have some ...