A stochastic, or random, process describes the correlation or evolution of random events. It is used to model stock market fluctuations and electronic/audio-visual/biological signals. Among the most well-known stochastic processes are random walks and Brownian motion.

learn more… | top users | synonyms

31
votes
7answers
5k views

Proving that 1- and 2-d simple symmetric random walks return to the origin with probability 1

How does one prove that a simple (steps of length 1 in directions parallel to the axes) symmetric (each possible direction is equally likely) random walk in 1 or 2 dimensions returns to the origin ...
3
votes
2answers
887 views

Transformation on state-space that preserves Markov property

I am solving a problem in Mathematical Statistics by Jun Shao Let $\{X_n \}$ be a Markov chain. Show that if $g$ is a one-to-one Borel function, then $\{g(X_n )\}$ is also a Markov chain. ...
5
votes
2answers
1k views

Time until a consecutive sequence of ones in a random bit sequence

This a reformulation of a practical problem I encountered. Say we have an infinite sequence of random, i.i.d bits. For each bit $X_i$, $P(X_i=1)=p$. What is the expected time until we get a sequence ...
3
votes
2answers
275 views

Using Recursion to Solve Coupon Collector

I read a brilliant answer by Mike Spivey on one of the questions and I was wondering how I could use it to solve a coupon collectors problem. The problem is : There are coupons labelled 1,2,3...,10 ...
1
vote
1answer
198 views

When random walk is upper unbounded

Consider a random walk $S_n = a_1+\dots+a_n$ where $a_n$ are iid random variables with $Ea_1 = a$ and $E|a_1|<\infty$. I am interested in the case when $\sup\limits_n S_n>M$ for all $M$ a.s. ...
12
votes
1answer
1k views

What is meant by a continuous-time white noise process?

What is meant by a continuous-time white noise process? In a discussion following a question a few months ago, I stated that as an engineer, I am used to thinking of a continuous-time ...
5
votes
2answers
3k views

Equilibrium distributions of Markov Chains

I often get confused about when a Markov chain has an equilibrium distribution; when this equilibrium distribution is unique; which starting states converge to the equilibrium distribution; and ...
3
votes
2answers
3k views

How to characterize recurrent and transient states of Markov chain

According to Wikipedia with a little rephrasing: A state $i$ is transient if and only if $P(T_i < \infty) <1$, recurrent if and only if $P(T_i < \infty) =1$, where $T_i$ is the ...
5
votes
2answers
401 views

Logarithm of a Markov Matrix

Start with a Markov matrix $\mathbf{M}$, whose elements are all between $0 \le \mathbf{M}_{ij} \le 1$ and each row sums to one. There is a natural connection with this matrix and the rate matrix ...
1
vote
1answer
168 views

Markov processes driven by the noise

Let $\xi_n\in \Xi$ be a sequence of iid random variables with $n \in\mathbb N\cup\{0\}$, which we call a noise process. Construct a process $$ Z_{n+1} = f(Z_n,\xi_n)\quad(\star) $$ with $Z_0\in E$ ...
1
vote
1answer
2k views

Relation between independent increments and Markov property

Independent increments and Markov property.do not imply each other. I was wondering if being one makes a process closer to being the other? if there are cases where one implies the other? ...
7
votes
1answer
3k views

equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution

I was wondering if equilibrium distribution, steady-state distribution, stationary distribution and limiting distribution mean the same, or there are differences between? I learn them in the context ...
4
votes
1answer
279 views

Feller continuity of the stochastic kernel

Given a metric space $X$ with a Borel sigma-algebra, the stochastic kernel $K(x,B)$ is such that $x\mapsto K(x,B)$ is a measurable function and a $B\mapsto K(x,B)$ is a probability measure on $X$ for ...
2
votes
1answer
569 views

Is continuous L2 bounded local martingale a true martingale?

I can prove it briefly, but I found a "counter" example. (There must be a mistake in the following words...) I can prove: X is a continuous local martingale, with $X_0=0$ a.s, then X is $L_2$ bounded ...
1
vote
2answers
142 views

How to model multi-step cell differentiation

Can I better explain cell lineages using PDEs or stochastic?
8
votes
2answers
2k views

Nice references on Markov chains/processes?

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes ...
9
votes
4answers
1k views

Showing that Brownian motion is bounded with non-zero probability

How do you show, that for every bound $\epsilon$, there is a non-zero probability that the motion is bounded on a finite interval. i.e. $$\mathbb{P} (\sup_{t\in[0,1]} |B(t)| < \epsilon) > 0$$ I ...
6
votes
1answer
2k views

Interpretation of sigma algebra

My question is how to interpret sigma algebra, especially in the context of probability theory (stochastic processes included). I would like to know if there is some clear and general way to interpret ...
5
votes
3answers
429 views

How to calculate $E[(\int_0^t{W_sds})^n], n \geq 2$

Let $W_t$ be a standard one dimension Brownian Motion with $W_0=0$ and $X_t=\int_0^t{W_sds}$. With the help of ito formula, we could get $$E[(X_t)^2]=\frac{1}{3}t^3$$ $$E[(X_t)^3]=0$$ When I try to ...
5
votes
3answers
3k views

What is the difference between all types of Markov Chains?

I have been looking for some good material covering Markov Chains but everything seems so difficult to me... After reading about the subject, I figured out that there is basically three kinds of ...
7
votes
1answer
2k views

Distribution of hitting time of line by Brownian motion

I came across the following question: Let $T_{a,b}$ denote the first hitting time of the line $a + bs$ by a standard Brownian motion, where $a > 0$ and $−\infty < b < \infty$ and let $T_a ...
1
vote
0answers
62 views

Estimation of a Ito's semi-martingale linear functional

Could someone check my solution for the following problem please? Or maybe propose a smarter/shorter solution. Consider a stochastic process $X=(X_t)_{t \in [0,1]}$ defined in a filtred ...
10
votes
1answer
312 views

Is a Markov process a random dynamic system?

A random dynamic system is defined in Wikipedia. Its definition, which is not included in this post for the sake of clarity, reminds me how similar a Markov process is to a random dynamic system just ...
9
votes
2answers
497 views

Joint moments of Brownian motion

My approach to this SE question uses the following joint moments of Brownian motion. For $n=1,2$ they are obvious and well-known, the others are not terribly hard to work out. Is there a reference ...
7
votes
2answers
604 views

Prokhorov metric vs. total variation norm

Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ ...
4
votes
2answers
292 views

A probability question

Suppose $X_1, X_2, ...,$ are IID random variables with $P(X_n=1)=p$ and $P(X_n=2)=1-p$. Let $S_n=\sum_{i=1}^n X_i$. I was wondering how to find $P(S_n \neq z, \forall n \in \mathbb{N})$ for some ...
4
votes
3answers
3k views

Sum and product of Martingale processes

Given two Martingale processes $(X_t)$ and $(Y_t)$, are their sum $(X_t+Y_t)$ and their product $(X_t \times Y_t)$ also Martingale? If not, will the two $(X_t)$ and $(Y_t)$ being independent grant ...
3
votes
1answer
967 views

Calculating stationary distribution of markov chain

I am asked to compute the stationary distribution of the markov chain with state space $E=\{0\dots,n\}$ and transition matrix below: \begin{bmatrix} 0 & 1 \\ \frac{1}{n} ...
7
votes
1answer
135 views

Must $n$ independent Wiener processes be simultaneously positive at some time?

Consider $n$ independent one-dimensional Wiener processes $(W_i)_{1\leqslant i\leqslant n}$. Is there with probability $1$ some time $t\in[0,1]$ such that $W_i(t)>0$ for every $1\leqslant ...
5
votes
2answers
538 views

Dominated convergence problems with Wald's identity for the Brownian Motion

In the course of proving Wald's second identity $E(B^2_T)=E(T)$, where $(B_t)_{t\geq0}$ is the Brownian motion and $T$ is a stopping time with $E(T)<\infty$, I got stuck with the following problem. ...
5
votes
1answer
271 views

Right continuous version of a martingale

This is an exercise in chapter 2 of the book "Continuous Martingales and Brownian Motion" by Revuz and Yor: Consider the probability space $([0,1], \mathcal{B}([0,1]), dx)$, where $dx$ denotes ...
4
votes
1answer
345 views

Applications of algebra and/or topology to stochastic (or Markov) processes

Some time back I was reading a PDF about algebra or topology (or algebraic topology, I forget which) and found an extremely enlightening section about an application to stochastic processes. ...
3
votes
2answers
934 views

A question regarding the hitting time formula in brownian motion

Let $\tau_a=\inf\{t: B_t=a\}$, the hitting time of the standard Brownian motion to reach the boundary $a$. This is easily derived $$E(e^{-\lambda \tau_a})=e^{-|a|\sqrt{2\lambda}}$$ But I am having ...
2
votes
2answers
165 views

First hitting time for a brownian motion with a exponential boundary

Let $B_t$ be the standard Brownian Motion. Is the distribution/density of the first hitting time of $B_t$ for an exponential decaying boundary known? Trying to be more formal, if ...
2
votes
1answer
1k views

covariance function for Brownian motion

What would the covariance function be of $V(t) = (1-t) B[t/(1-t)]$ if $B(t)$ is standard Brownian motion. Also $t$ is between $0$ and $1$. Thanks for the help! EDIT: Here is where I am stuck: I ...
0
votes
2answers
1k views

Pure Birth Process Question

I would appreciate any possible help for this question because I have no clue what to do! Thanks so much! Consider a population made of a fixed number (N) of people. At time t=0 there is only one ...
9
votes
2answers
711 views

Random walk on n-cycle

For a graph G, let W be the (random) vertex occupied at the first time the random walk has visited every vertex. That is, W is the last new vertex to be visited by the random walk. Prove the following ...
7
votes
1answer
270 views

PDE - Feynman-Kac vs. finite difference methods

I've heard that in greater than three dimensions, it's more efficient to solve a second-order parabolic PDE using a Monte-Carlo method based on the Feynman-Kac formula that it is to use finite ...
5
votes
2answers
2k views

How do you check if a sequence of numbers is truly random? [duplicate]

Suppose a source produces an indefinite sequence of positive integers. How can you check whether the numbers are generated truly randomly?
4
votes
1answer
395 views

Doob's inequality in probability

In the book "Optimal Stopping and Free-Boundary Problems" there is given Doob's inequality of the following form. Let $X = (X_t,F_t)$ be a submartingale. Then for any $\varepsilon>0$ and each ...
3
votes
2answers
243 views

How can a Markov chain be written as a measure-preserving dynamic system

From http://masi.cscs.lsa.umich.edu/~crshalizi/notabene/ergodic-theory.html irreducible Markov chains with finite state spaces are ergodic processes, since they have a unique invariant ...
3
votes
1answer
701 views

Why does a time-homogeneous Markov process possess the Markov property?

Klenke defines (Definition 17.3, p. 346) a time-homogeneous Markov process independently, rather than as a special case of a stochastic process that possesses the Markov property (Definition 17.1, p. ...
3
votes
1answer
580 views

Relations between Order Statistics of Uniform RVs and Exponential RVs

Say we have $U_1 \dots U_n$ i.i.d. random variables uniform on $[0,1]$ and $Y_1 \dots Y_{n+1}$ i.i.d. random variables distributed as $Y_i \sim Exp(1)$. I know that the joint distribution of the order ...
3
votes
1answer
348 views

Random walk $< 0$

Suppose ${X_t}$ is a random walk with mean zero. (either discrete or continuous time) Fix a time $T$. What is: $P[X_t < 0 \text{ for all } t \leq T]$? In words, what's the probability the random ...
2
votes
1answer
90 views

solution of SDE: $dS_t=(\alpha S_t+f(t))dW_t$

does someone know how to solve the following SDE $$dS_t=(\alpha S_t+f(t))dW_t, S_0=s$$ where $f(t)$ is a deterministic function and $W_t$ is a standard brownian motion. Is there a explicit solution ...
2
votes
1answer
422 views

Limiting distribution and initial distribution of a Markov chain

For a Markov chain (can the following discussion be for either discrete time or continuous time, or just discrete time?), if for an initial distribution i.e. the distribution of $X_0$, there exists ...
2
votes
1answer
197 views

Homogeneous Poisson processes

Suppose that $N$ is a homogeneous Poisson process with rate $\lambda$. For $0 \le s \le t < \infty$, how can we find $\mathbb E[N_s\cdot N_t]$?
2
votes
2answers
122 views

sub martingales and more

This is a problem on sub-martingales. Given : $X_n = X_0 \mathrm{e}^{\mu S_n}$, $n= 1,2,3,\ldots$, where $X_0 > 0$ and where $S_n$ is a symmetric random walk and $\mu$ is greater than zero. We ...
2
votes
1answer
330 views

Probability distribution of sign changes in Brownian motion

Let us consider a 1d Brownian motion. Displacements in space will be positive or negative and this is a random variable $U(t)$ that characterizes a random process and that can take just the values ...
1
vote
0answers
68 views

Differential equation with random variable

How can I derive analytically or compute numerically the solution to following differential equation $$ dy/dt = y\cdot X\cdot (y\cdot X - g(y,X))\cdot X $$ where X is a random variable (e.g. from a ...