A stochastic, or random, process describes the correlation or evolution of random events. It is used to model stock market fluctuations and electronic/audio-visual/biological signals. Among the most well-known stochastic processes are random walks and Brownian motion.

learn more… | top users | synonyms

32
votes
7answers
5k views

Proving that 1- and 2-d simple symmetric random walks return to the origin with probability 1

How does one prove that a simple (steps of length 1 in directions parallel to the axes) symmetric (each possible direction is equally likely) random walk in 1 or 2 dimensions returns to the origin ...
3
votes
2answers
1k views

Transformation on state-space that preserves Markov property

I am solving a problem in Mathematical Statistics by Jun Shao Let $\{X_n \}$ be a Markov chain. Show that if $g$ is a one-to-one Borel function, then $\{g(X_n )\}$ is also a Markov chain. ...
5
votes
2answers
1k views

Time until a consecutive sequence of ones in a random bit sequence

This a reformulation of a practical problem I encountered. Say we have an infinite sequence of random, i.i.d bits. For each bit $X_i$, $P(X_i=1)=p$. What is the expected time until we get a sequence ...
6
votes
2answers
3k views

Equilibrium distributions of Markov Chains

I often get confused about when a Markov chain has an equilibrium distribution; when this equilibrium distribution is unique; which starting states converge to the equilibrium distribution; and ...
1
vote
1answer
176 views

Markov processes driven by the noise

Let $\xi_n\in \Xi$ be a sequence of iid random variables with $n \in\mathbb N\cup\{0\}$, which we call a noise process. Construct a process $$ Z_{n+1} = f(Z_n,\xi_n)\quad(\star) $$ with $Z_0\in E$ ...
3
votes
2answers
296 views

Using Recursion to Solve Coupon Collector

I read a brilliant answer by Mike Spivey on one of the questions and I was wondering how I could use it to solve a coupon collectors problem. The problem is : There are coupons labelled 1,2,3...,10 ...
1
vote
1answer
209 views

When random walk is upper unbounded

Consider a random walk $S_n = a_1+\dots+a_n$ where $a_n$ are iid random variables with $Ea_1 = a$ and $E|a_1|<\infty$. I am interested in the case when $\sup\limits_n S_n>M$ for all $M$ a.s. ...
12
votes
1answer
1k views

What is meant by a continuous-time white noise process?

What is meant by a continuous-time white noise process? In a discussion following a question a few months ago, I stated that as an engineer, I am used to thinking of a continuous-time ...
6
votes
1answer
3k views

Distribution of hitting time of line by Brownian motion

I came across the following question: Let $T_{a,b}$ denote the first hitting time of the line $a + bs$ by a standard Brownian motion, where $a > 0$ and $−\infty < b < \infty$ and let $T_a ...
4
votes
2answers
4k views

How to characterize recurrent and transient states of Markov chain

According to Wikipedia with a little rephrasing: A state $i$ is transient if and only if $P(T_i < \infty) <1$, recurrent if and only if $P(T_i < \infty) =1$, where $T_i$ is the ...
8
votes
2answers
698 views

Prokhorov metric vs. total variation norm

Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ ...
5
votes
2answers
417 views

Logarithm of a Markov Matrix

Start with a Markov matrix $\mathbf{M}$, whose elements are all between $0 \le \mathbf{M}_{ij} \le 1$ and each row sums to one. There is a natural connection with this matrix and the rate matrix ...
1
vote
1answer
2k views

Relation between independent increments and Markov property

Independent increments and Markov property.do not imply each other. I was wondering if being one makes a process closer to being the other? if there are cases where one implies the other? ...
7
votes
1answer
3k views

equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution

I was wondering if equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution mean the same, or there are differences between? I learn them in the context ...
4
votes
1answer
294 views

Feller continuity of the stochastic kernel

Given a metric space $X$ with a Borel sigma-algebra, the stochastic kernel $K(x,B)$ is such that $x\mapsto K(x,B)$ is a measurable function and a $B\mapsto K(x,B)$ is a probability measure on $X$ for ...
2
votes
1answer
647 views

Is continuous L2 bounded local martingale a true martingale?

I can prove it briefly, but I found a "counter" example. (There must be a mistake in the following words...) I can prove: X is a continuous local martingale, with $X_0=0$ a.s, then X is $L_2$ bounded ...
1
vote
2answers
145 views

How to model multi-step cell differentiation

Can I better explain cell lineages using PDEs or stochastic?
8
votes
2answers
2k views

Nice references on Markov chains/processes?

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes ...
9
votes
4answers
1k views

Showing that Brownian motion is bounded with non-zero probability

How do you show, that for every bound $\epsilon$, there is a non-zero probability that the motion is bounded on a finite interval. i.e. $$\mathbb{P} (\sup_{t\in[0,1]} |B(t)| < \epsilon) > 0$$ I ...
6
votes
1answer
2k views

Interpretation of sigma algebra

My question is how to interpret sigma algebra, especially in the context of probability theory (stochastic processes included). I would like to know if there is some clear and general way to interpret ...
5
votes
3answers
446 views

How to calculate $E[(\int_0^t{W_sds})^n], n \geq 2$

Let $W_t$ be a standard one dimension Brownian Motion with $W_0=0$ and $X_t=\int_0^t{W_sds}$. With the help of ito formula, we could get $$E[(X_t)^2]=\frac{1}{3}t^3$$ $$E[(X_t)^3]=0$$ When I try to ...
5
votes
3answers
3k views

What is the difference between all types of Markov Chains?

I have been looking for some good material covering Markov Chains but everything seems so difficult to me... After reading about the subject, I figured out that there is basically three kinds of ...
7
votes
1answer
855 views

Recommendation on stochastic process books

I was wondering if someone could recommend good books on stochastic processes with measure theory treatment with not much or no measure theory treatment for each, it would be nice to have some ...
3
votes
1answer
1k views

Calculating stationary distribution of markov chain

I am asked to compute the stationary distribution of the markov chain with state space $E=\{0\dots,n\}$ and transition matrix below: \begin{bmatrix} 0 & 1 \\ \frac{1}{n} ...
1
vote
0answers
68 views

Estimation of a Ito's semi-martingale linear functional

Could someone check my solution for the following problem please? Or maybe propose a smarter/shorter solution. Consider a stochastic process $X=(X_t)_{t \in [0,1]}$ defined in a filtred ...
10
votes
1answer
348 views

Is a Markov process a random dynamic system?

A random dynamic system is defined in Wikipedia. Its definition, which is not included in this post for the sake of clarity, reminds me how similar a Markov process is to a random dynamic system just ...
9
votes
2answers
549 views

Joint moments of Brownian motion

My approach to this SE question uses the following joint moments of Brownian motion. For $n=1,2$ they are obvious and well-known, the others are not terribly hard to work out. Is there a reference ...
5
votes
3answers
3k views

Sum and product of Martingale processes

Given two Martingale processes $(X_t)$ and $(Y_t)$, are their sum $(X_t+Y_t)$ and their product $(X_t \times Y_t)$ also Martingale? If not, will the two $(X_t)$ and $(Y_t)$ being independent grant ...
4
votes
2answers
295 views

A probability question

Suppose $X_1, X_2, ...,$ are IID random variables with $P(X_n=1)=p$ and $P(X_n=2)=1-p$. Let $S_n=\sum_{i=1}^n X_i$. I was wondering how to find $P(S_n \neq z, \forall n \in \mathbb{N})$ for some ...
7
votes
1answer
141 views

Must $n$ independent Wiener processes be simultaneously positive at some time?

Consider $n$ independent one-dimensional Wiener processes $(W_i)_{1\leqslant i\leqslant n}$. Is there with probability $1$ some time $t\in[0,1]$ such that $W_i(t)>0$ for every $1\leqslant ...
5
votes
2answers
573 views

Dominated convergence problems with Wald's identity for the Brownian Motion

In the course of proving Wald's second identity $E(B^2_T)=E(T)$, where $(B_t)_{t\geq0}$ is the Brownian motion and $T$ is a stopping time with $E(T)<\infty$, I got stuck with the following problem. ...
5
votes
1answer
290 views

Right continuous version of a martingale

This is an exercise in chapter 2 of the book "Continuous Martingales and Brownian Motion" by Revuz and Yor: Consider the probability space $([0,1], \mathcal{B}([0,1]), dx)$, where $dx$ denotes ...
4
votes
1answer
377 views

Applications of algebra and/or topology to stochastic (or Markov) processes

Some time back I was reading a PDF about algebra or topology (or algebraic topology, I forget which) and found an extremely enlightening section about an application to stochastic processes. ...
3
votes
3answers
510 views

Wiener Process $dB^2=dt$

Why is $dB^2=dt$? Every online source I've come across lists this as an exercise or just states it, but why isn't this ever explicitly proved? I know that $dB=\sqrt{dt}Z$, but I don't know what ...
3
votes
2answers
1k views

A question regarding the hitting time formula in brownian motion

Let $\tau_a=\inf\{t: B_t=a\}$, the hitting time of the standard Brownian motion to reach the boundary $a$. This is easily derived $$E(e^{-\lambda \tau_a})=e^{-|a|\sqrt{2\lambda}}$$ But I am having ...
2
votes
2answers
238 views

First hitting time for a brownian motion with a exponential boundary

Let $B_t$ be the standard Brownian Motion. Is the distribution/density of the first hitting time of $B_t$ for an exponential decaying boundary known? Trying to be more formal, if ...
2
votes
1answer
1k views

covariance function for Brownian motion

What would the covariance function be of $V(t) = (1-t) B[t/(1-t)]$ if $B(t)$ is standard Brownian motion. Also $t$ is between $0$ and $1$. Thanks for the help! EDIT: Here is where I am stuck: I ...
0
votes
2answers
1k views

Pure Birth Process Question

I would appreciate any possible help for this question because I have no clue what to do! Thanks so much! Consider a population made of a fixed number (N) of people. At time t=0 there is only one ...
9
votes
2answers
769 views

Random walk on n-cycle

For a graph G, let W be the (random) vertex occupied at the first time the random walk has visited every vertex. That is, W is the last new vertex to be visited by the random walk. Prove the following ...
7
votes
1answer
291 views

PDE - Feynman-Kac vs. finite difference methods

I've heard that in greater than three dimensions, it's more efficient to solve a second-order parabolic PDE using a Monte-Carlo method based on the Feynman-Kac formula that it is to use finite ...
5
votes
2answers
2k views

How do you check if a sequence of numbers is truly random? [duplicate]

Suppose a source produces an indefinite sequence of positive integers. How can you check whether the numbers are generated truly randomly?
4
votes
1answer
411 views

Doob's inequality in probability

In the book "Optimal Stopping and Free-Boundary Problems" there is given Doob's inequality of the following form. Let $X = (X_t,F_t)$ be a submartingale. Then for any $\varepsilon>0$ and each ...
3
votes
2answers
280 views

How can a Markov chain be written as a measure-preserving dynamic system

From http://masi.cscs.lsa.umich.edu/~crshalizi/notabene/ergodic-theory.html irreducible Markov chains with finite state spaces are ergodic processes, since they have a unique invariant ...
3
votes
1answer
875 views

Why does a time-homogeneous Markov process possess the Markov property?

Klenke defines (Definition 17.3, p. 346) a time-homogeneous Markov process independently, rather than as a special case of a stochastic process that possesses the Markov property (Definition 17.1, p. ...
3
votes
1answer
622 views

Relations between Order Statistics of Uniform RVs and Exponential RVs

Say we have $U_1 \dots U_n$ i.i.d. random variables uniform on $[0,1]$ and $Y_1 \dots Y_{n+1}$ i.i.d. random variables distributed as $Y_i \sim Exp(1)$. I know that the joint distribution of the order ...
3
votes
1answer
365 views

Random walk $< 0$

Suppose ${X_t}$ is a random walk with mean zero. (either discrete or continuous time) Fix a time $T$. What is: $P[X_t < 0 \text{ for all } t \leq T]$? In words, what's the probability the random ...
2
votes
1answer
103 views

solution of SDE: $dS_t=(\alpha S_t+f(t))dW_t$

does someone know how to solve the following SDE $$dS_t=(\alpha S_t+f(t))dW_t, S_0=s$$ where $f(t)$ is a deterministic function and $W_t$ is a standard brownian motion. Is there a explicit solution ...
2
votes
1answer
461 views

Limiting distribution and initial distribution of a Markov chain

For a Markov chain (can the following discussion be for either discrete time or continuous time, or just discrete time?), if for an initial distribution i.e. the distribution of $X_0$, there exists ...
2
votes
1answer
203 views

Homogeneous Poisson processes

Suppose that $N$ is a homogeneous Poisson process with rate $\lambda$. For $0 \le s \le t < \infty$, how can we find $\mathbb E[N_s\cdot N_t]$?
2
votes
2answers
125 views

sub martingales and more

This is a problem on sub-martingales. Given : $X_n = X_0 \mathrm{e}^{\mu S_n}$, $n= 1,2,3,\ldots$, where $X_0 > 0$ and where $S_n$ is a symmetric random walk and $\mu$ is greater than zero. We ...