Questions tagged [hidden-markov-models]

This tag is for questions relating to "Hidden Markov model", a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobservable (i.e. hidden) states.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Hidden Markov Model of a stationary Markov chain is a stationary process.

Let $X=(X_i)_{i \ge 1}$ be an irreducible Markov chain started in its stationary distribution, and $Y=(Y_i)_{i \ge 1}$ be such that $Y_i=\phi(X_i)$ for an arbitrary function $\phi$. Note that $X$ is a ...
hegash's user avatar
  • 181
-1 votes
0 answers
23 views

Help with this backwards Markov kernel setup in Chopin and Papaspiliopoulos

Here's the problem, as far as I understand it. For given measure spaces: $$(\mathcal{X}_0, B(\mathcal{X}_0), \mathbb{P}_0)$$ $$(\mathcal{X}_1, B(\mathcal{X}_1), \text{don't-care-about-this-measure-yet?...
Jason Cole's user avatar
0 votes
0 answers
25 views

EM algorithm for Markov switching models.

Consider the model $y_t = F_{S_t} x_t + \varepsilon_{S_t}$ and $x_t = A_{S_t} x_{t-1} + \nu_{S_t}$, where $\varepsilon_{S_t}, \nu_{S_t} \sim N(0, R_{S_t})$ and $N(0, Q_{S_t})$ and $S_t$ is Markov ...
openspace's user avatar
  • 6,470
0 votes
1 answer
48 views

Likelihood computation for hidden markov models.

If we have a $2$-state model (i.e. the simplest non-trivial example) in a hidden markov model, and some generated observation-data $\mathcal{O}$ from the algorithm for generating observations. Is it ...
Ben123's user avatar
  • 952
0 votes
0 answers
29 views

Baum-Welch Algorithm

Let $S = \{S_1,\ldots,S_N\}$ be the number of states of a hidden markov process, and let $V = \{V_1,\ldots,V_N\}$ be the alphabet of possible observations one can make. Given a model $\lambda = (\pi,A,...
Ben123's user avatar
  • 952
0 votes
0 answers
11 views

When do mixtures of ergodic Markov kernels remain ergodic?

Given two Markov kernels on the same space $\mathfrak X$ and relative to the same dominating measure, $K_0(\cdot,\cdot)$ and $K_1(\cdot,\cdot)$, both ergodic with respective stationary distribution ...
Xi'an ні війні's user avatar
0 votes
0 answers
33 views

Marginal probabilities from hidden Markov model transition matrix

I have hidden Markov model transition matrix, $T$, defined as: \begin{bmatrix} \lambda & 1 - \lambda \\\ 0 & 1 \end{bmatrix} I know that each row, $j$, and column, $k$, of $T$ provides the ...
user_15's user avatar
  • 73
0 votes
0 answers
18 views

Hidden Markov Model that is trained on and emits probability vectors as observed variables

I am currently working on a project, in which I aim to fit a model to a series of observed data points. These data points consist of probability vectors. Each vector adds up to a value of 1. E.g. ...
DanTheMan's user avatar
-1 votes
1 answer
49 views

Hidden Markov Model - Observations [closed]

I'm struggling to understand the statistic relation referring the observations of an HMM. That's clear for me: Output Independence My problem is how this equation can be derived: probability of an ...
olympus_mons's user avatar
0 votes
1 answer
30 views

First-passage time distribution in Laplace space?

I'm struggling to understand the reasoning between moving between two steps in a reaction scheme for a paper I am reading. For this (from the description), the probabilities over different paths are ...
JHS's user avatar
  • 3
0 votes
0 answers
47 views

Russell & Norvig: Connecting models of probabilistic reasoning to Stochastic Differential Equations

Artificial Intelligence: A Modern Approach, 4th Global ed. by Stuart Russell and Peter Norvig contains the following footnote on page 480 of chapter 14: Uncertainty over continuous time can be ...
BrentKylling's user avatar
0 votes
0 answers
12 views

Log probability of following a trajectory under an optimal policy

In reinforcement learning, is the log probability of following a trajectory under an optimal policy equal to the sum of rewards for that trajectory? i.e. $\log(p(\tau)) = \sum^T_{t=1}r(s_t,a_t)$ I've ...
Nicholas James Bailey's user avatar
0 votes
0 answers
24 views

Average of Hidden-Markov Model emission pairs: convergence proof

I'm running into a practical problem and figured the best way to solve it would be to ask around as I feel I've done a conceptual mistake somewhere which eludes me. I apologize as such: to make it ...
Magic Biscuit's user avatar
0 votes
0 answers
32 views

Rigorous derivation of the joint density of the state space model

This is a definition of a state-space model given in An Introduction to Sequential Monte Carlo by Chopin and Papaspiliopoulos. A state-space model is a time series model that consists of two discrete-...
nomadicmathematician's user avatar
1 vote
0 answers
53 views

Derivation of Viterbi Algorithm

I am trying to understand the derivation of the Viterbi algorithm for hidden Markov models. I understand that the motivation is to find the maximum probability path estimate, i.e., \begin{equation} ...
Otto's user avatar
  • 13
0 votes
1 answer
29 views

Hidden Markov Models with Finite State Space and Infinite Observation Space

How could I rewrite the following definition of a hidden Markov model for a finite signal state space $E$ and an infinite observation state space $F$? Or if both were finite? A stochastic process $(X_{...
Otto's user avatar
  • 13
0 votes
1 answer
67 views

A city-traveller problem with unknown connections

Some time ago I took part in an entry mathematics exam to a masters programme of one of the universities in my country. I solved most of the tasks, but the last one. Since the admission committee ...
o.spectrum's user avatar
  • 1,170
0 votes
0 answers
27 views

On change of measure of hidden Markov model (discrete states and observations)

We consider a system whose state is described by a finite-state, homogeneous, discrete-time Markove chain $X_k$, $k\in \Bbb N$. $X_0$ is given. Suppose the state space of $X_k$ is $S_X=\{e_1,\cdots ,...
Steven Lu's user avatar
  • 1,037
1 vote
0 answers
17 views

Expression cycle certainty as probability

If I know for a fact that event A happens 21 times before event B happens, and B then happens 7 times before event A and the cycle repeats, Can I say that the probability of the B happening after ...
TSR's user avatar
  • 507
1 vote
0 answers
30 views

Is HMM Smoothing more computationally complex than HMM filtering?

Given a sequence of observations $y_1, y_2, \dots, y_n$, of some lates sequence of Markovian states $x_1, x_2, \dots, x_n$: HMM filtering computes $p(x_k|y_1,\dots,y_k) \\$ HMM smoothing computes $p(...
Ben's user avatar
  • 138
0 votes
0 answers
33 views

Volatility matrix understanding

I have problem to understand volatility matrix on page 9 section 2.3 The Yield-Adjustment Term in the following article: Volatility matrix in Nelson-Siegel. It is volatility matrix in AFNS model. Why ...
Martin N.'s user avatar
  • 149
3 votes
0 answers
100 views

HMM, reverse engineering the transition matrix

I fitted a 2-states-HMM model last week, and generate a bunch of 1s and 0s, but I forgot to store its parameters (transition matrix). Now, I only got these 1s and 0s, how do I backward/reverse-...
kou's user avatar
  • 159
1 vote
0 answers
53 views

Viterbi algorithm for object-tracking

I have a sequence of images, and I need to find and track the creation of the objects, then their movement and then their disappearance. There can be up to $3$ objects overall, and sometimes there are ...
josh92's user avatar
  • 11
0 votes
0 answers
20 views

Where does discrete probabilities in Forward-Backward algorithm for Hidden Markov Models come from?

I am trying to derive a Forward-Backward algorithm used in Hidden Markov Models to compute the likelihood $P(x | \theta)$ that sample $x = (x_1, ... x_n)$ comes from HMM defined by set of parameters $\...
Brzoskwinia's user avatar
2 votes
1 answer
51 views

States and observations in a HMM for FX markets

I would like to set up a hidden Markov model (HMM) for foreign exchange (FX) markets. To start with, I am thinking of a model that only has three states "up", "down" and "flat&...
Thomas Hemming Larsen's user avatar
2 votes
0 answers
51 views

An application of Birkhoff's ergodic theorem to Hidden Markov Models

Let $(X_{n}, Y_{n})_{n\in\mathbb{Z}}$ be a hidden Markov Model where the $(Y_{n})_{n}$ are the observations and the $(X_{n})_{n}$ are the hidden states which only take values 0 or 1. Assume that the ...
Ibra's user avatar
  • 130
1 vote
0 answers
34 views

Problems understanding the mathematical notation of the Forward algorithm in Hidden Markov Models

I found a recursive version of the forward algorithm on wikipedia, however I don't understand the notation given in the pseudocode: What means $$x_{t-1}$$ under the summation sign? What do I need to ...
teoML's user avatar
  • 111
2 votes
1 answer
293 views

HMM - conditional probabilities

I am learning Hidden Markov Model, and I have some trouble to understand how the independance is used in the calculus \begin{aligned} \mathbb{P}(O(t) \mid y(t), \lambda) &=\prod_{j=1}^{\ell} \...
Takamovic's user avatar
0 votes
1 answer
25 views

Inference in state space models

I have the following state space model $$x_{n}=x_{n-1}+cos(1.2n)$$ $$y_{n}=x_{n}^{2}+w_{n}$$ $$w_{n}\thicksim N(0,σ_{w}^{2})$$ For the observation pdf, we have $y_{n}\thicksim N(x_{n},{x_{n}σ_{w}}^{2})...
Maria K's user avatar
0 votes
1 answer
312 views

Probability of observing sequence Markov model

I have been trying to understand hidden Markov models with observational probability but I often find myself confused. I have discussed with my tutor for further help however, he is often rude and ...
ASH's user avatar
  • 147
1 vote
0 answers
23 views

A question on Cox-Processes (Markov-Modulated Poisson processes)

I´m trying to prove that a Markov-Modulated Poisson process could be seen as a 2-dimensional continuous time markov chain. For this, I'm considering a two state markov chain $\{J(t) : t \geq 0\}$ with ...
Maruchan01's user avatar
0 votes
0 answers
25 views

a question about the direction of a flow of numbers

I am trying to figure out if a stream of numbers i say more positive or more negative or neutral. Let's say I have a stream of numbers that I can sample. I would like to estimate are these last x ...
sean's user avatar
  • 1
1 vote
1 answer
62 views

On the equivalence of different assumptions of Hidden Markov Models (HMM)

I am currently studying Hidden Markov Models (HMM). We denote the hidden quantities as $(X_0, \dots, X_n) \in \mathcal{X}^{n+1}$ and the observed quantities $(Y_1, \dots, Y_n) \in \mathcal{Y}^n$. The ...
rkvymvqt's user avatar
0 votes
0 answers
39 views

POMDPs: How to update transition probabilities after receiving new information

I am trying to model a POMDP based on user feedback dialogue and an observation set based on eye gaze. The goal is to specify which item the robot should pick up according which item the human's eye ...
chelzo's user avatar
  • 23
1 vote
0 answers
130 views

Shortest Path using Markov Decision

For the following question, one solves the problem to find the shortest path from the Start to the End in the following maze using the Markov decision process. We formulate this problem as the ...
BlizzardWalker's user avatar
1 vote
0 answers
74 views

Construction of Markov Chain

Let's consider a toy example of applying a Markov Chain Monte Carlo (MCMC) method. Let $X$ be a discrete random variable whose unnormalized probability mass function is $$ \tilde{p}(X=1)=4, \tilde{p}(...
BlizzardWalker's user avatar
3 votes
1 answer
220 views

Hidden Markov Model

I am reading "Bayesian Reasoning And Machine Learning" and I'm doing exercise 23.3 (a) on p.490. Here's the exercise: Consider a HMM with 3 states $(M=3)$ and $2$ output symbols, with a ...
Slim Shady's user avatar
1 vote
1 answer
60 views

$\beta$-recursion calculation in matrix form in Hidden Markov Models

In the backward algorithm for inference in hidden markov models, how would we calculate $\beta$ in matrix form? In the answer to this question, I saw how to calculate $\alpha$ in matrix form, and now ...
Slim Shady's user avatar
1 vote
0 answers
19 views

Differenciate between two distributions using gibbs sampling

For $t=1,\dots, n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl} \sigma^2 & \text{with probability} & p\\ 1 & \text{with probability} & 1-p \...
Abdoul Haki's user avatar
2 votes
1 answer
312 views

Hidden Markov Model - why backward probability is conditional on the current state

I'm trying to understand hidden Markov model (HMM). Here is the material which I studied. It states that there are two assumptions in HMM (page 3): $P( q_i | q_1, ..., q_{i-1} ) = P( q_i | q_{i-1} )$ ...
Chun-Ye Lu's user avatar
1 vote
1 answer
204 views

Hidden Markov model - Probability of arbitrary (start and length) state sequence given all observations?

Consider a hidden Markov model $\lambda=\{A,B,q\}$, where $q$ is the initial probability matrix, $A$ the transition probability matrix, and $B$ the output probability distributions. Assume that we ...
Defindun's user avatar
  • 185
1 vote
2 answers
100 views

Bibliography on phylogenetic trees from a mathematical point of view

I am learning about phylogenetic trees, but it is difficult for me to find some documents/books focused on the mathematics. I found Pachter & Sturmfels's article1, which is quite good, to be ...
Mart's user avatar
  • 11
0 votes
0 answers
76 views

What's the application of doubly-stochastic matrices in engineering?

Today I learned the existence of such matrix . wolframe This is indeed a very interesting thing. I am wondering if there is any realworld application of such matrix. It seems that this matrix has ...
user152503's user avatar
0 votes
0 answers
15 views

How should forward-backward algorithm be implemented for a text file with multiple twitter posts?

Assuming i have a textfile which shows multiple twitter posts made by different users, how do we approach the forward-backward algorithm? Do we apply throughout the entire textfile by treating it as ...
Alex Dogan's user avatar
1 vote
0 answers
53 views

Expectations in Hidden Markov Models

Background: Suppose I have an HMM characterized by the parameters $\theta = (\mathbf{A}, \mathbf{B}, \mathbf{\pi})$, where $A,B,\pi$ are the hidden state transition probability matrix, the emission ...
The Dude's user avatar
  • 1,079
0 votes
1 answer
163 views

Markov Chain: Conditional distribution at time $t$, given $t-1$ and $t+1$

For a Markov process given by $$x_t = \mu +\kappa(x_{t-1} - \mu) + \sigma \cdot \varepsilon_t $$ where $\varepsilon_t \sim N(0,1)$ and $\mu$, $\kappa$, $\sigma^2$ are the parameters, how would I find ...
Heung Min Son's user avatar
1 vote
1 answer
57 views

Performing Inference on Hidden Markov Models with GMM Emission Probabilities

So I have a hidden markov model with two hidden states $z = a$ and $z = b$. My emission probabilities are given by: $$ P\left( x_{n} \mid z = a \right) = \frac{\pi_{1}}{\pi_{1} + \pi_{2}} \mathcal{N}(...
The Dude's user avatar
  • 1,079
0 votes
1 answer
98 views

Transformations of stochastic matrix that preserve equilibrium

I have a stochastic (Markov) matrix $W$. I would like to modify it, such that $W_{i,i}$ increases for all $i$ (and thus other elements decrease). However, I don't want to change the equilibrium ...
mbarete's user avatar
  • 113
1 vote
0 answers
42 views

Measurability question about measure dependent on variables

Suppose $(Y, {\cal T}, \nu)$ is a measure space and $(X, {\cal S}, \mu_y)$ is a measure space for each fixed $y \in Y$; $\mu_y$ depends on $y$. What can we say about the ${\cal T}-{\cal B}$ ...
Trash Failure's user avatar
0 votes
1 answer
267 views

Hidden Markov Model - understanding Viterbi algorithm

I try to understand the Viterbi algorithm for solving hidden Markov models. There is a pseudo-code of it in Wikipedia: In the row that marked in blue (starts with $T_2$) I don't understand: how does ...
S. R's user avatar
  • 368