0
votes
1answer
20 views

Steady state of a $4 \times 4$ transition matrix

Normally I just take $q(M_{m\times n} - I_{m\times n})$ to workout the steady state, but here I have: $$\left(\begin{array}{rrrr} 0 & 0 & .8 & .2 \\ .4 & .6 & 0 & 0 \\ .2 ...
0
votes
1answer
17 views

Limiting probability of Markov chain(Terminology)

If I am asked to find the limiting probability of a Markov chain, what does this pertain to? $\lim \limits_{n \to \infty} P^n$? Where $P$ is the stepping matrix and $n$ is the number of steps. "What ...
0
votes
0answers
56 views

Financial mathematic with Feynman-Kac

I have a really big task in financial mathematics and a small part of it (to set up the problem), I need to write a PIDE (the Feynman-kac) where we estimate options with jumps. It is derived from the ...
0
votes
0answers
64 views

Markov Chains Proof using Statistics

Source: This came from "Introduction to probability" by Charles Miller Grinstead, and James Aurie Snell. It was located on page 407 and is Theorem 11.1 in the section 11.1 Introduction. Theorem: The ...
0
votes
1answer
74 views

How to Prove by definition, the given process is a Markov Process?

Define the process Xt by X0 = 1, and for t = 1, 2, . . . Xt = { uXt-1, with probability p, { vXt-1, with probability 1-p where 0 < v < 1 ...
0
votes
0answers
45 views

Propagation of standard deviation for random variable with Markov Property

I have a discrete random variable, $X \in \{0,1,2,3\}$. Define the indicator function: $$ 1_{k}\left(x\right) = \begin{cases} 1, & \text{if $x=k$} \\ 0, & \text{otherwise} \\ \end{cases}$$ ...
3
votes
1answer
67 views

A equivalent definition of the Feller Process.

I saw this on Liggett's Book (P.95). Let $S=% %TCIMACRO{\U{2115} }% %BeginExpansion \mathbb{N} %EndExpansion ,$ and suppose $\left( X_{t}\right) _{t\geq 0}$ is a continuous-time Markov process with ...
1
vote
1answer
68 views

A question about Infinitesimal generator of Feller Process

Let $S=% %TCIMACRO{\U{211d} }% %BeginExpansion \mathbb{R} %EndExpansion $, and consider the Feller process $\left( X_{t}\right) _{t\geq 0}$ with state space $S$ such that $X_{t}=t+X_{0}$ for all ...
1
vote
0answers
71 views

Conditional distributions of (higher-order) autoregressive Markov processes

If we specify an $p$-th order autoregressive process in discrete time by its transition distribution $F_{t|t-1,\ldots,t-p}$, what can be said about lower order conditional distribution where we ...
2
votes
1answer
70 views

local variance of Markov decision processes

Does anybody know the notion of "local variance" of Markov decision processes? Any reference would be appreciated. Thanks.
2
votes
2answers
169 views

Markov chain basic positive recurrency question

If a discrete markov chain is stationary (as far as I know: doesn't modify itself with time), irreducible (doesn't have transient states) and aperiodic (no periodic states), is it positive recurrent? ...