A stochastic, or random, process describes the correlation or evolution of random events. It is used to model stock market fluctuations and electronic/audio-visual/biological signals. Among the most well-known stochastic processes are random walks and Brownian motion.

learn more… | top users | synonyms

35
votes
7answers
8k views

Proving that 1- and 2-d simple symmetric random walks return to the origin with probability 1

How does one prove that a simple (steps of length $1$ in directions parallel to the axes) symmetric (each possible direction is equally likely) random walk in $1$ or $2$ dimensions returns to the ...
7
votes
2answers
4k views

Equilibrium distributions of Markov Chains

I often get confused about when a Markov chain has an equilibrium distribution; when this equilibrium distribution is unique; which starting states converge to the equilibrium distribution; and ...
4
votes
2answers
2k views

Transformation on state-space that preserves Markov property

I am solving a problem in Mathematical Statistics by Jun Shao Let $\{X_n \}$ be a Markov chain. Show that if $g$ is a one-to-one Borel function, then $\{g(X_n )\}$ is also a Markov chain. ...
4
votes
2answers
405 views

Using Recursion to Solve Coupon Collector

I read a brilliant answer by Mike Spivey on one of the questions and I was wondering how I could use it to solve a coupon collectors problem. The problem is : There are coupons labelled 1,2,3...,10 ...
11
votes
1answer
3k views

Interpretation of sigma algebra

My question is how to interpret sigma algebra, especially in the context of probability theory (stochastic processes included). I would like to know if there is some clear and general way to interpret ...
9
votes
1answer
5k views

Distribution of hitting time of line by Brownian motion

I came across the following question: Let $T_{a,b}$ denote the first hitting time of the line $a + bs$ by a standard Brownian motion, where $a > 0$ and $−\infty < b < \infty$ and let $T_a ...
5
votes
2answers
2k views

Time until a consecutive sequence of ones in a random bit sequence

This a reformulation of a practical problem I encountered. Say we have an infinite sequence of random, i.i.d bits. For each bit $X_i$, $P(X_i=1)=p$. What is the expected time until we get a sequence ...
5
votes
3answers
751 views

Wiener Process $dB^2=dt$

Why is $dB^2=dt$? Every online source I've come across lists this as an exercise or just states it, but why isn't this ever explicitly proved? I know that $dB=\sqrt{dt}Z$, but I don't know what ...
1
vote
2answers
3k views

First exit time for Brownian motion without drift

I am dealing with the simulation of particles exhibiting Brownian motion without drift, currently by updating the position in given time steps $\Delta t$ by random displacement in each direction drawn ...
5
votes
2answers
509 views

Logarithm of a Markov Matrix

Start with a Markov matrix $\mathbf{M}$, whose elements are all between $0 \le \mathbf{M}_{ij} \le 1$ and each row sums to one. There is a natural connection with this matrix and the rate matrix ...
1
vote
1answer
214 views

Markov processes driven by the noise

Let $\xi_n\in \Xi$ be a sequence of iid random variables with $n \in\mathbb N\cup\{0\}$, which we call a noise process. Construct a process $$ Z_{n+1} = f(Z_n,\xi_n)\quad(\star) $$ with $Z_0\in E$ ...
8
votes
1answer
5k views

equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution

I was wondering if equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution mean the same, or there are differences between? I learn them in the context ...
1
vote
1answer
243 views

When random walk is upper unbounded

Consider a random walk $S_n = a_1+\dots+a_n$ where $a_n$ are iid random variables with $Ea_1 = a$ and $E|a_1|<\infty$. I am interested in the case when $\sup\limits_n S_n>M$ for all $M$ a.s. ...
0
votes
1answer
100 views

Prove $\mathbb{P}(\sup_{t \geq 0} M_t > x \mid \mathcal{F}_0)= 1 \wedge \frac{M_0}{x}$ for a martingale $(M_t)_{t \geq 0}$

Let $M$ be a positive, continuous martingale that converges a.s. to zero as $t$ tends to infinity. I now want to prove that for every $x>0$ $$ P\left( \sup_{t \geq 0 } M_t > x \mid \mathcal{F}_0 ...
60
votes
4answers
4k views

Why did my friend lose all his money?

Not sure if this is a question for math.se or stats.se, but here we go: Our MUD (Multi-User-Dungeon, a sort of textbased world of warcraft) has a casino where players can play a simple roulette. My ...
14
votes
1answer
2k views

What is meant by a continuous-time white noise process?

What is meant by a continuous-time white noise process? In a discussion following a question a few months ago, I stated that as an engineer, I am used to thinking of a continuous-time ...
12
votes
3answers
387 views

Limit of a Wiener integral

How to show that $$ \lim _{\alpha \rightarrow \infty } \sup_{t \in \left [0,T \right]} \left | e^{-\alpha t} \int _ 0 ^t e^{\alpha s} ~ dB_s \right | =0, \ \ \text{a.e.} $$ where $\left (B_s ...
5
votes
2answers
5k views

How to characterize recurrent and transient states of Markov chain

According to Wikipedia with a little rephrasing: A state $i$ is transient if and only if $P(T_i < \infty) <1$, recurrent if and only if $P(T_i < \infty) =1$, where $T_i$ is the ...
9
votes
2answers
725 views

Joint moments of Brownian motion

My approach to this SE question uses the following joint moments of Brownian motion. For $n=1,2$ they are obvious and well-known, the others are not terribly hard to work out. Is there a reference ...
8
votes
2answers
1k views

Prokhorov metric vs. total variation norm

Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ ...
4
votes
1answer
221 views

“Continuity” of stochastic integral wrt Brownian motion

I'd like to prove a nice property of a stochastic integral with respect to Brownian motion. Let $(H_t)_{t\geq0}$ be a progressive and bounded process that is continuous at $0$ and $B$ a standard ...
6
votes
1answer
516 views

Applications of algebra and/or topology to stochastic (or Markov) processes

Some time back I was reading a PDF about algebra or topology (or algebraic topology, I forget which) and found an extremely enlightening section about an application to stochastic processes. ...
4
votes
0answers
157 views

6-digit password - a special decoding method

Consider the situation of decoding a 6-digit password that consists of the symbols A to Z and 0 to 9, where all possible combinations are tried randomly and uniformly. Consider the ...
2
votes
1answer
2k views

Relation between independent increments and Markov property

Independent increments and Markov property.do not imply each other. I was wondering if being one makes a process closer to being the other? if there are cases where one implies the other? ...
5
votes
1answer
330 views

Feller continuity of the stochastic kernel

Given a metric space $X$ with a Borel sigma-algebra, the stochastic kernel $K(x,B)$ is such that $x\mapsto K(x,B)$ is a measurable function and a $B\mapsto K(x,B)$ is a probability measure on $X$ for ...
2
votes
1answer
962 views

Is continuous L2 bounded local martingale a true martingale?

I can prove it briefly, but I found a "counter" example. (There must be a mistake in the following words...) I can prove: X is a continuous local martingale, with $X_0=0$ a.s, then X is $L_2$ bounded ...
1
vote
2answers
154 views

How to model multi-step cell differentiation

Can I better explain cell lineages using PDEs or stochastic?
13
votes
2answers
3k views

Nice references on Markov chains/processes?

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes ...
12
votes
4answers
2k views

Showing that Brownian motion is bounded with non-zero probability

How do you show, that for every bound $\epsilon$, there is a non-zero probability that the motion is bounded on a finite interval. i.e. $$\mathbb{P} (\sup_{t\in[0,1]} |B(t)| < \epsilon) > 0$$ I ...
8
votes
3answers
1k views

Why is stopping time defined as a random variable?

I've been given a crash course in stochastic processes and martingales for the purposes of a semester project on them. The guy I'm working with has been, I feel, a little vague in the definition of ...
6
votes
3answers
5k views

What is the difference between all types of Markov Chains?

I have been looking for some good material covering Markov Chains but everything seems so difficult to me... After reading about the subject, I figured out that there is basically three kinds of ...
7
votes
1answer
1k views

Recommendation on stochastic process books

I was wondering if someone could recommend good books on stochastic processes with measure theory treatment with not much or no measure theory treatment for each, it would be nice to have some ...
5
votes
3answers
494 views

How to calculate $E[(\int_0^t{W_sds})^n], n \geq 2$

Let $W_t$ be a standard one dimension Brownian Motion with $W_0=0$ and $X_t=\int_0^t{W_sds}$. With the help of ito formula, we could get $$E[(X_t)^2]=\frac{1}{3}t^3$$ $$E[(X_t)^3]=0$$ When I try to ...
3
votes
1answer
390 views

Hermite Polynomials and Brownian motion

I am asked to prove the following : Let $B_t$ be a standard brownian motion. The $n$th Hermite polynomial is $\displaystyle H_n(t,x)=\frac{(−t)^n}{n!} e^{x^2/(2t)} \frac{d^n}{dx^n}e^{-x^2/(2t)}$. ...
12
votes
2answers
1k views

Random walk on n-cycle

For a graph G, let W be the (random) vertex occupied at the first time the random walk has visited every vertex. That is, W is the last new vertex to be visited by the random walk. Prove the following ...
12
votes
2answers
1k views

Quadratic variation of Brownian motion and almost-sure convergence

Say that $W(t)$ is a Brownian motion. The quadratic variation $[W,W](t)$ is defined in terms of a partition $\Pi = \{0 = t_0 < t_1 < \cdots < t_n = t\}$ by $$ \begin{split} [W,W](t) &= ...
3
votes
1answer
2k views

Calculating stationary distribution of markov chain

I am asked to compute the stationary distribution of the markov chain with state space $E=\{0\dots,n\}$ and transition matrix below: \begin{bmatrix} 0 & 1 \\ \frac{1}{n} ...
1
vote
0answers
88 views

Estimation of a Ito's semi-martingale linear functional

Could someone check my solution for the following problem please? Or maybe propose a smarter/shorter solution. Consider a stochastic process $X=(X_t)_{t \in [0,1]}$ defined in a filtred ...
9
votes
1answer
469 views

Is a Markov process a random dynamic system?

A random dynamic system is defined in Wikipedia. Its definition, which is not included in this post for the sake of clarity, reminds me how similar a Markov process is to a random dynamic system just ...
6
votes
2answers
489 views

Laplace transform of integrated geometric Brownian motion

Is there any closed form of the Laplace transform of an integrated geometric Brownian motion ? A geometric Brownian motion $X=(X_t)_{t \geq 0}$ satisifies $dX_t = \sigma X_t \, dW_t$ where ...
5
votes
1answer
371 views

Right continuous version of a martingale

This is an exercise in chapter 2 of the book "Continuous Martingales and Brownian Motion" by Revuz and Yor: Consider the probability space $([0,1], \mathcal{B}([0,1]), dx)$, where $dx$ denotes ...
5
votes
3answers
4k views

Sum and product of Martingale processes

Given two Martingale processes $(X_t)$ and $(Y_t)$, are their sum $(X_t+Y_t)$ and their product $(X_t \times Y_t)$ also Martingale? If not, will the two $(X_t)$ and $(Y_t)$ being independent grant ...
4
votes
2answers
302 views

A probability question

Suppose $X_1, X_2, ...,$ are IID random variables with $P(X_n=1)=p$ and $P(X_n=2)=1-p$. Let $S_n=\sum_{i=1}^n X_i$. I was wondering how to find $P(S_n \neq z, \forall n \in \mathbb{N})$ for some ...
3
votes
0answers
55 views

No drift brownian motion problem

Given two same brownian motion with no drift and different variances: $$dG_1= \sigma_1 G_1 dW $$ $$dG_2= \sigma_2 G_2 dW $$ and two barriers $P_1 > P_2$ assuming that $ \sigma_1 > \sigma_2 $ ...
2
votes
2answers
624 views

First hitting time for a brownian motion with a exponential boundary

Let $B_t$ be the standard Brownian Motion. Is the distribution/density of the first hitting time of $B_t$ for an exponential decaying boundary known? Trying to be more formal, if ...
1
vote
2answers
164 views

Survival probability up to time $n$ in a branching process.

Let $\{Z_n : n=0,1,2,\ldots\}$ be a Galton-Watson branching process with time-homogeneous offspring distribution $$\mathbb P(Z_{n,j} = 0) = 1-p = 1 - \mathbb P(Z_{n,j}=2), $$ where $0<p<1$. That ...
20
votes
7answers
9k views

What's the difference between stochastic and random?

What's the difference between stochastic and random? I've read in the portuguese wikipedia that there's a difference, but I still didn't see this point on english wikipedia.
7
votes
1answer
145 views

Must $n$ independent Wiener processes be simultaneously positive at some time?

Consider $n$ independent one-dimensional Wiener processes $(W_i)_{1\leqslant i\leqslant n}$. Is there with probability $1$ some time $t\in[0,1]$ such that $W_i(t)>0$ for every $1\leqslant ...
6
votes
1answer
581 views

Hölder Continuity of Fractional Brownian Motion

I would like to prove the following theorem: Let $H\in (0,1)$. The fractional Brownian motion $B_H$ admits a version whose sample paths are $a.s.$ Hölder continuous of order strict less than $H$. ...
6
votes
1answer
558 views

stopped filtration = filtration generated by stopped process?

I am interested in a proof of the following statement which seems intuitive, but is somehow really tricky: Let $X$ be a stochastic process and let $(\mathcal{F}(t) : t \geq 0)$ be the filtration ...