A stochastic, or random, process describes the correlation or evolution of random events. It is used to model stock market fluctuations and electronic/audio-visual/biological signals. Among the most well-known stochastic processes are random walks and Brownian motion.

learn more… | top users | synonyms

40
votes
7answers
10k views

Proving that 1- and 2-d simple symmetric random walks return to the origin with probability 1

How does one prove that a simple (steps of length $1$ in directions parallel to the axes) symmetric (each possible direction is equally likely) random walk in $1$ or $2$ dimensions returns to the ...
2
votes
1answer
93 views

When Superposition of Two Renewal Processes is another Renewal Process?

When superposition of two renewal processes is another renewal process? If you merge (superpose) two Poisson processes with parameters $\lambda_1$ and $\lambda_2$, the outcome is another Poisson ...
5
votes
3answers
919 views

Wiener Process $dB^2=dt$

Why is $dB^2=dt$? Every online source I've come across lists this as an exercise or just states it, but why isn't this ever explicitly proved? I know that $dB=\sqrt{dt}Z$, but I don't know what ...
9
votes
1answer
6k views

Distribution of hitting time of line by Brownian motion

I came across the following question: Let $T_{a,b}$ denote the first hitting time of the line $a + bs$ by a standard Brownian motion, where $a > 0$ and $−\infty < b < \infty$ and let $T_a ...
5
votes
2answers
2k views

Time until a consecutive sequence of ones in a random bit sequence

This a reformulation of a practical problem I encountered. Say we have an infinite sequence of random, i.i.d bits. For each bit $X_i$, $P(X_i=1)=p$. What is the expected time until we get a sequence ...
14
votes
2answers
2k views

Random walk on $n$-cycle

For a graph $G$, let $W$ be the (random) vertex occupied at the first time the random walk has visited every vertex. That is, $W$ is the last new vertex to be visited by the random walk. Prove the ...
7
votes
2answers
4k views

Equilibrium distributions of Markov Chains

I often get confused about when a Markov chain has an equilibrium distribution; when this equilibrium distribution is unique; which starting states converge to the equilibrium distribution; and ...
4
votes
2answers
471 views

Using Recursion to Solve Coupon Collector

I read a brilliant answer by Mike Spivey on one of the questions and I was wondering how I could use it to solve a coupon collectors problem. The problem is : There are coupons labelled 1,2,3...,10 ...
3
votes
1answer
1k views

Angle bracket and sharp bracket for discontinuous processes

The question is quite simple actually. I am trying to understand the differences between the angle bracket $\left<X,Y\right>$ of two processes with jumps $X,Y$, and the sharp bracket of $[X,Y]$. ...
1
vote
1answer
248 views

Markov processes driven by the noise

Let $\xi_n\in \Xi$ be a sequence of iid random variables with $n \in\mathbb N\cup\{0\}$, which we call a noise process. Construct a process $$ Z_{n+1} = f(Z_n,\xi_n)\quad(\star) $$ with $Z_0\in E$ ...
0
votes
1answer
119 views

Prove $\mathbb{P}(\sup_{t \geq 0} M_t > x \mid \mathcal{F}_0)= 1 \wedge \frac{M_0}{x}$ for a martingale $(M_t)_{t \geq 0}$

Let $M$ be a positive, continuous martingale that converges a.s. to zero as $t$ tends to infinity. I now want to prove that for every $x>0$ $$ P\left( \sup_{t \geq 0 } M_t > x \mid \mathcal{F}_0 ...
18
votes
1answer
4k views

Interpretation of sigma algebra

My question is how to interpret sigma algebra, especially in the context of probability theory (stochastic processes included). I would like to know if there is some clear and general way to interpret ...
12
votes
3answers
2k views

Why is stopping time defined as a random variable?

I've been given a crash course in stochastic processes and martingales for the purposes of a semester project on them. The guy I'm working with has been, I feel, a little vague in the definition of ...
16
votes
2answers
2k views

What is meant by a continuous-time white noise process?

What is meant by a continuous-time white noise process? In a discussion following a question a few months ago, I stated that as an engineer, I am used to thinking of a continuous-time ...
13
votes
3answers
415 views

Limit of a Wiener integral

How to show that $$ \lim _{\alpha \rightarrow \infty } \sup_{t \in \left [0,T \right]} \left | e^{-\alpha t} \int _ 0 ^t e^{\alpha s} ~ dB_s \right | =0, \ \ \text{a.e.} $$ where $\left (B_s ...
2
votes
2answers
4k views

First exit time for Brownian motion without drift

I am dealing with the simulation of particles exhibiting Brownian motion without drift, currently by updating the position in given time steps $\Delta t$ by random displacement in each direction drawn ...
5
votes
2answers
565 views

Logarithm of a Markov Matrix

Start with a Markov matrix $\mathbf{M}$, whose elements are all between $0 \le \mathbf{M}_{ij} \le 1$ and each row sums to one. There is a natural connection with this matrix and the rate matrix ...
10
votes
1answer
6k views

equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution

I was wondering if equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution mean the same, or there are differences between? I learn them in the context ...
5
votes
3answers
584 views

Difference between Modification and Indistinguishable

Would someone be able to offer a layman's explanation of what is means when two stochastic processes are a Modification of each other and when they are Indistinguishable? My Stochastic Analysis notes ...
1
vote
1answer
276 views

When random walk is upper unbounded

Consider a random walk $S_n = a_1+\dots+a_n$ where $a_n$ are iid random variables with $Ea_1 = a$ and $E|a_1|<\infty$. I am interested in the case when $\sup\limits_n S_n>M$ for all $M$ a.s. ...
0
votes
1answer
50 views

Asymmetric Random Walk / Prove that $T:= \inf\{n: X_n = b\}$ is a $\{\mathscr F_n\}_{n \in \mathbb N}$-stopping time

Given random variables $Y_1, Y_2, ... \stackrel{iid}{\sim} P(Y_i = 1) = p = 1 - q = 1 - P(Y_i = -1)$ where $p > q$ in a filtered probability space $(\Omega, \mathscr F, \{\mathscr F_n\}_{n \in ...
0
votes
1answer
200 views

Prove Z is a martingale by defining it is a product of random variables

Given a filtered probability space $(\Omega, \mathscr{F}, \{\mathscr{F_n}\}, \mathbb{P})$ where $\mathscr{F_n} = \mathscr{F_n}^{Z} \doteq \sigma(Z_0, Z_1, \ldots, Z_n)$, show that $Z = (Z_n)_{n \geq ...
3
votes
1answer
433 views

Random walk $< 0$

Suppose ${X_t}$ is a random walk with mean zero. (either discrete or continuous time) Fix a time $T$. What is: $P[X_t < 0 \text{ for all } t \leq T]$? In words, what's the probability the random ...
62
votes
4answers
4k views

Why did my friend lose all his money?

Not sure if this is a question for math.se or stats.se, but here we go: Our MUD (Multi-User-Dungeon, a sort of textbased world of warcraft) has a casino where players can play a simple roulette. My ...
5
votes
2answers
6k views

How to characterize recurrent and transient states of Markov chain

According to Wikipedia with a little rephrasing: A state $i$ is transient if and only if $P(T_i < \infty) <1$, recurrent if and only if $P(T_i < \infty) =1$, where $T_i$ is the ...
9
votes
2answers
829 views

Joint moments of Brownian motion

My approach to this SE question uses the following joint moments of Brownian motion. For $n=1,2$ they are obvious and well-known, the others are not terribly hard to work out. Is there a reference ...
6
votes
1answer
435 views

Right continuous version of a martingale

This is an exercise in chapter 2 of the book "Continuous Martingales and Brownian Motion" by Revuz and Yor: Consider the probability space $([0,1], \mathcal{B}([0,1]), dx)$, where $dx$ denotes ...
2
votes
1answer
3k views

Relation between independent increments and Markov property

Independent increments and Markov property.do not imply each other. I was wondering if being one makes a process closer to being the other? if there are cases where one implies the other? ...
8
votes
2answers
1k views

Prokhorov metric vs. total variation norm

Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ ...
4
votes
1answer
255 views

“Continuity” of stochastic integral wrt Brownian motion

I'd like to prove a nice property of a stochastic integral with respect to Brownian motion. Let $(H_t)_{t\geq0}$ be a progressive and bounded process that is continuous at $0$ and $B$ a standard ...
7
votes
1answer
658 views

Applications of algebra and/or topology to stochastic (or Markov) processes

Some time back I was reading a PDF about algebra or topology (or algebraic topology, I forget which) and found an extremely enlightening section about an application to stochastic processes. ...
6
votes
1answer
347 views

Feller continuity of the stochastic kernel

Given a metric space $X$ with a Borel sigma-algebra, the stochastic kernel $K(x,B)$ is such that $x\mapsto K(x,B)$ is a measurable function and a $B\mapsto K(x,B)$ is a probability measure on $X$ for ...
4
votes
0answers
206 views

6-digit password - a special decoding method

Consider the situation of decoding a 6-digit password that consists of the symbols A to Z and 0 to 9, where all possible combinations are tried randomly and uniformly. Consider the ...
2
votes
1answer
1k views

Is continuous L2 bounded local martingale a true martingale?

I can prove it briefly, but I found a "counter" example. (There must be a mistake in the following words...) I can prove: X is a continuous local martingale, with $X_0=0$ a.s, then X is $L_2$ bounded ...
1
vote
2answers
159 views

How to model multi-step cell differentiation

Can I better explain cell lineages using PDEs or stochastic?
14
votes
2answers
4k views

Nice references on Markov chains/processes?

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes ...
13
votes
4answers
2k views

Showing that Brownian motion is bounded with non-zero probability

How do you show, that for every bound $\epsilon$, there is a non-zero probability that the motion is bounded on a finite interval. i.e. $$\mathbb{P} (\sup_{t\in[0,1]} |B(t)| < \epsilon) > 0$$ I ...
13
votes
3answers
3k views

The Laplace transform of the first hitting time of Brownian motion

Let $B_t$ be the standard Brownian motion process, $a > 0$, and let $H_a = \inf \{ t : B_t > a \}$ be a stopping time. I want to show that the Laplace transform of $H_a$ is ...
6
votes
3answers
6k views

What is the difference between all types of Markov Chains?

I have been looking for some good material covering Markov Chains but everything seems so difficult to me... After reading about the subject, I figured out that there is basically three kinds of ...
8
votes
1answer
2k views

Recommendation on stochastic process books

I was wondering if someone could recommend good books on stochastic processes with measure theory treatment with not much or no measure theory treatment for each, it would be nice to have some ...
7
votes
1answer
186 views

When does $\sum_{i=1}^{\infty} X_i$ exist for random sequences $\{X_i\}_{i=1}^{\infty}$?

Suppose $\{X_1, X_2, X_3, \ldots\}$ is an infinite sequence of random variables such that $E[X_i]=0$ for all $i$, and $E[X_iX_j]=0$ whenever $i \neq j$. Further suppose the variances $\sigma_i^2 = ...
22
votes
8answers
12k views

What's the difference between stochastic and random?

What's the difference between stochastic and random? I've read in the Portuguese Wikipedia that there's a difference, but I still didn't see this point on English Wikipedia.
5
votes
3answers
513 views

How to calculate $E[(\int_0^t{W_sds})^n], n \geq 2$

Let $W_t$ be a standard one dimension Brownian Motion with $W_0=0$ and $X_t=\int_0^t{W_sds}$. With the help of ito formula, we could get $$E[(X_t)^2]=\frac{1}{3}t^3$$ $$E[(X_t)^3]=0$$ When I try to ...
3
votes
1answer
534 views

Hermite Polynomials and Brownian motion

I am asked to prove the following : Let $B_t$ be a standard brownian motion. The $n$th Hermite polynomial is $\displaystyle H_n(t,x)=\frac{(−t)^n}{n!} e^{x^2/(2t)} \frac{d^n}{dx^n}e^{-x^2/(2t)}$. ...
12
votes
2answers
1k views

Quadratic variation of Brownian motion and almost-sure convergence

Say that $W(t)$ is a Brownian motion. The quadratic variation $[W,W](t)$ is defined in terms of a partition $\Pi = \{0 = t_0 < t_1 < \cdots < t_n = t\}$ by $$ \begin{split} [W,W](t) &= ...
10
votes
1answer
544 views

Is a Markov process a random dynamic system?

A random dynamic system is defined in Wikipedia. Its definition, which is not included in this post for the sake of clarity, reminds me how similar a Markov process is to a random dynamic system just ...
7
votes
3answers
5k views

Sum and product of Martingale processes

Given two Martingale processes $(X_t)$ and $(Y_t)$, are their sum $(X_t+Y_t)$ and their product $(X_t \times Y_t)$ also Martingale? If not, will the two $(X_t)$ and $(Y_t)$ being independent grant ...
3
votes
1answer
199 views

Is a probability of 0 or 1 given information up to time t unchanged by information thereafter?

Given a filtered probability space $(\Omega, \mathscr{F}, \{\mathscr{F}_n\}_{n \in \mathbb{N}}, \mathbb{P})$, let $A \in \mathscr{F}$. Suppose $$\exists t \in \mathbb{N} \ \text{s.t.} \ E[1_A | ...
3
votes
2answers
180 views

The strong Markov property with an uncountable index set

The following definition of the strong Markov property, from Klenke's book, supposes an index set $I$ that is not necessarily countable. However, it is explicitly mentioned previously (following Lemma ...
3
votes
1answer
2k views

Calculating stationary distribution of markov chain

I am asked to compute the stationary distribution of the markov chain with state space $E=\{0\dots,n\}$ and transition matrix below: \begin{bmatrix} 0 & 1 \\ \frac{1}{n} ...