Questions tagged [hidden-markov-models]
This tag is for questions relating to "Hidden Markov model", a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobservable (i.e. hidden) states.
80
questions
0
votes
0
answers
19
views
Emulating any Markov Process of N states using a restricted Markov Process of more than N states
Let's define an unrestricted family of Markov Processes on $N$ states as the set of all possible Markov Processes using the states $1, 2, ... N$.
To clarify, if we let $A$ be the current state and $B$ ...
- 31
-1
votes
0
answers
12
views
Is there any references for solving inverse Ising problem w.r.t. some objective functions other than MaxLikelihood
I am trying to formulate an inverse Ising problem that optimizes some defined objective functions other than maximum likelihood. I am pretty new to this field (only some background on Markov random ...
1
vote
0
answers
15
views
Is HMM Smoothing more computationally complex than HMM filtering?
Given a sequence of observations $y_1, y_2, \dots, y_n$, of some lates sequence of Markovian states $x_1, x_2, \dots, x_n$:
HMM filtering computes $p(x_k|y_1,\dots,y_k) \\$
HMM smoothing computes $p(...
- 118
0
votes
0
answers
9
views
viterbi algorithm with non-repeating states
I'd like to use the viterbi algorithm to get the MAP sequence of hidden states, but I would like to add a constraint: reject sequences in which hidden states repeat. Anyone knows how that should be ...
0
votes
0
answers
25
views
Volatility matrix understanding
I have problem to understand volatility matrix on page 9 section 2.3 The Yield-Adjustment Term in the following article: Volatility matrix in Nelson-Siegel. It is volatility matrix in AFNS model. Why ...
- 149
0
votes
0
answers
12
views
Hidden Markov Model Sequence Prediction
I am trying to predict the state of rain based on observed rainfall in centimeters. The three states are ' little rain' 'some rain' and 'a lot of rain'. For the prediction, when I enter the amount of ...
- 103
0
votes
0
answers
18
views
Viterbi algorithm differs between digital communications and more general HMM?
I posted this to the stats Stack
Exchange about 1.5
weeks ago, but haven't gotten a response.
I am self-learning Markov modelling, currently looking at simple
examples of hidden markov models (HMMs) ...
- 189
3
votes
0
answers
87
views
HMM, reverse engineering the transition matrix
I fitted a 2-states-HMM model last week, and generate a bunch of 1s and 0s, but I forgot to store its parameters (transition matrix). Now, I only got these 1s and 0s, how do I backward/reverse-...
- 159
1
vote
0
answers
48
views
Viterbi algorithm for object-tracking
I have a sequence of images, and I need to find and track the creation of the objects, then their movement and then their disappearance. There can be up to $3$ objects overall, and sometimes there are ...
- 11
0
votes
0
answers
16
views
Where does discrete probabilities in Forward-Backward algorithm for Hidden Markov Models come from?
I am trying to derive a Forward-Backward algorithm used in Hidden Markov Models to compute the likelihood $P(x | \theta)$ that sample $x = (x_1, ... x_n)$ comes from HMM defined by set of parameters $\...
- 241
0
votes
0
answers
52
views
Mixture of Markov Models question
This is a follow up from this question.
Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
- 29
2
votes
0
answers
42
views
An application of Birkhoff's ergodic theorem to Hidden Markov Models
Let $(X_{n}, Y_{n})_{n\in\mathbb{Z}}$ be a hidden Markov Model where the $(Y_{n})_{n}$ are the observations and the $(X_{n})_{n}$ are the hidden states which only take values 0 or 1. Assume that the ...
- 78
1
vote
0
answers
23
views
Problems understanding the mathematical notation of the Forward algorithm in Hidden Markov Models
I found a recursive version of the forward algorithm on wikipedia, however I don't understand the notation given in the pseudocode:
What means $$x_{t-1}$$ under the summation sign? What do I need to ...
- 111
2
votes
1
answer
117
views
HMM - conditional probabilities
I am learning Hidden Markov Model, and I have some trouble to understand how the independance is used in the calculus
\begin{aligned}
\mathbb{P}(O(t) \mid y(t), \lambda) &=\prod_{j=1}^{\ell} \...
- 21
0
votes
1
answer
23
views
Inference in state space models
I have the following state space model
$$x_{n}=x_{n-1}+cos(1.2n)$$
$$y_{n}=x_{n}^{2}+w_{n}$$
$$w_{n}\thicksim N(0,σ_{w}^{2})$$
For the observation pdf, we have $y_{n}\thicksim N(x_{n},{x_{n}σ_{w}}^{2})...
- 3
0
votes
1
answer
145
views
Probability of observing sequence Markov model
I have been trying to understand hidden Markov models with observational probability but I often find myself confused. I have discussed with my tutor for further help however, he is often rude and ...
- 147
1
vote
0
answers
23
views
A question on Cox-Processes (Markov-Modulated Poisson processes)
I´m trying to prove that a Markov-Modulated Poisson process could be seen as a 2-dimensional continuous time markov chain.
For this, I'm considering a two state markov chain $\{J(t) : t \geq 0\}$ with ...
- 11
0
votes
0
answers
25
views
a question about the direction of a flow of numbers
I am trying to figure out if a stream of numbers i say more positive or more negative or neutral.
Let's say I have a stream of numbers that I can sample. I would like to estimate are these last x ...
- 1
1
vote
1
answer
42
views
On the equivalence of different assumptions of Hidden Markov Models (HMM)
I am currently studying Hidden Markov Models (HMM). We denote the hidden quantities as $(X_0, \dots, X_n) \in \mathcal{X}^{n+1}$ and the observed quantities $(Y_1, \dots, Y_n) \in \mathcal{Y}^n$. The ...
- 45
0
votes
0
answers
22
views
POMDPs: How to update transition probabilities after receiving new information
I am trying to model a POMDP based on user feedback dialogue and an observation set based on eye gaze. The goal is to specify which item the robot should pick up according which item the human's eye ...
- 23
1
vote
0
answers
59
views
Shortest Path using Markov Decision
For the following question, one solves the problem to find the shortest path from the Start to the End in the following maze using the Markov decision process.
We formulate this problem as the ...
- 915
3
votes
0
answers
61
views
Construction of Markov Chain
Let's consider a toy example of applying a Markov Chain Monte Carlo (MCMC) method. Let $X$ be a discrete random variable whose unnormalized probability mass function is
$$
\tilde{p}(X=1)=4, \tilde{p}(...
- 915
3
votes
1
answer
103
views
Hidden Markov Model
I am reading "Bayesian Reasoning And Machine Learning" and I'm doing exercise 23.3 (a) on p.490.
Here's the exercise:
Consider a HMM with 3 states $(M=3)$ and $2$ output symbols, with a ...
- 353
1
vote
1
answer
47
views
$\beta$-recursion calculation in matrix form in Hidden Markov Models
In the backward algorithm for inference in hidden markov models, how would we calculate $\beta$ in matrix form? In the answer to this question, I saw how to calculate $\alpha$ in matrix form, and now ...
- 353
1
vote
0
answers
17
views
Differenciate between two distributions using gibbs sampling
For $t=1,\dots, n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl}
\sigma^2 & \text{with probability} & p\\
1 & \text{with probability} & 1-p
\...
- 464
2
votes
1
answer
163
views
Hidden Markov Model - why backward probability is conditional on the current state
I'm trying to understand hidden Markov model (HMM). Here is the material which I studied.
It states that there are two assumptions in HMM (page 3):
$P( q_i | q_1, ..., q_{i-1} ) = P( q_i | q_{i-1} )$
...
- 172
1
vote
1
answer
86
views
Hidden Markov model - Probability of arbitrary (start and length) state sequence given all observations?
Consider a hidden Markov model $\lambda=\{A,B,q\}$, where $q$ is the initial probability matrix, $A$ the transition probability matrix, and $B$ the output probability distributions.
Assume that we ...
- 175
0
votes
1
answer
38
views
Bibliography about Phylogenetic trees from a math point of view
I am learning about phylogenetic trees, but it is difficult for me to find some documents/books focused on the maths.
I saw this article https://www.researchgate.net/publication/...
- 1
0
votes
0
answers
42
views
What's the application of doubly-stochastic matrices in engineering?
Today I learned the existence of such matrix .
wolframe
This is indeed a very interesting thing. I am wondering if there is any realworld application of such matrix.
It seems that this matrix has ...
- 193
0
votes
0
answers
15
views
How should forward-backward algorithm be implemented for a text file with multiple twitter posts?
Assuming i have a textfile which shows multiple twitter posts made by different users, how do we approach the forward-backward algorithm? Do we apply throughout the entire textfile by treating it as ...
1
vote
0
answers
44
views
Expectations in Hidden Markov Models
Background:
Suppose I have an HMM characterized by the parameters $\theta = (\mathbf{A}, \mathbf{B}, \mathbf{\pi})$, where $A,B,\pi$ are the hidden state transition probability matrix, the emission ...
- 1,039
0
votes
1
answer
113
views
Markov Chain: Conditional distribution at time $t$, given $t-1$ and $t+1$
For a Markov process given by
$$x_t = \mu +\kappa(x_{t-1} - \mu) + \sigma \cdot \varepsilon_t $$
where $\varepsilon_t \sim N(0,1)$ and $\mu$, $\kappa$, $\sigma^2$ are the parameters, how would I find ...
1
vote
1
answer
46
views
Performing Inference on Hidden Markov Models with GMM Emission Probabilities
So I have a hidden markov model with two hidden states $z = a$ and $z = b$. My emission probabilities are given by:
$$ P\left( x_{n} \mid z = a \right) = \frac{\pi_{1}}{\pi_{1} + \pi_{2}} \mathcal{N}(...
- 1,039
0
votes
1
answer
62
views
Transformations of stochastic matrix that preserve equilibrium
I have a stochastic (Markov) matrix $W$. I would like to modify it, such that $W_{i,i}$ increases for all $i$ (and thus other elements decrease). However, I don't want to change the equilibrium ...
- 113
1
vote
0
answers
42
views
Measurability question about measure dependent on variables
Suppose $(Y, {\cal T}, \nu)$ is a measure space and $(X, {\cal S}, \mu_y)$ is a measure space for each fixed $y \in Y$; $\mu_y$ depends on $y$. What can we say about the ${\cal T}-{\cal B}$ ...
- 711
0
votes
1
answer
195
views
Hidden Markov Model - understanding Viterbi algorithm
I try to understand the Viterbi algorithm for solving hidden Markov models. There is a pseudo-code of it in Wikipedia:
In the row that marked in blue (starts with $T_2$) I don't understand: how does ...
- 338
0
votes
0
answers
23
views
HMMs: Difference between the joint and conditional probabilities
I am having trouble in giving meaning to the joint and conditional probabilities related to the observations and states of HMMs in the Appendix A of Speech and Language Processing by Jurafsky and ...
- 1
0
votes
0
answers
76
views
Unsure how to solve first-order Markov Chain problem
I am working on solving the following problem (I am new to Markov Chains):
-It can be either rainy or sunny on a given day.
-The probability that a rainy day is followed by a rainy day is 0.5.
-The ...
1
vote
0
answers
23
views
Limiting probabilities in Markov Decision Process
Do limiting probabilities exist for discrete-time Markov Decision Process, given that the actions are deterministic? If so, how should it be calculated?
Please provide links to notes/references, if ...
- 37
1
vote
0
answers
39
views
Best time to make a guess in an HMM with discounting?
Consider a hidden Markov model (HMM) with $2$ states and $2$ outputs. The transition probabilities are $p_{ij}$, $i,j=1,2$ and output (emission) probabilities are $q_{ik}$, $i=1,2$, $k=1,2$. Assume an ...
- 383
0
votes
1
answer
78
views
Three state Markov chain
If I have Transition matrix
$T=\begin{pmatrix}
2/3 &0 &1/3 \\
1/4 &3/4 &0 \\
1/3& 0 &2/3
\end{pmatrix}$
How would I get the quantity
$R_i^{(n)}=\sum_{k=1}^{n}(T^k)_{ii}$...
- 63
1
vote
0
answers
27
views
How to determine optimal state sequence in HMM?
There are several criteria to determine state sequences in HMM. For example, most possible state for each individual observation, most possible pair states, and most possible sequence.
Which one ...
- 123
1
vote
0
answers
30
views
partially observed hidden markov model
I am working on a data learning problem. Here's the framework: the data X_it for each observation i=1,...,N, time t=1,...,T, measures a bio-marker over time for an observation (continuous values). An ...
- 21
5
votes
1
answer
219
views
How to interpret clusters on Markov chain time characteristics?
I have a complex network $G=(V,E)$ from multivariate financial time series in which a single vertex $v_i$ represents the types of states corresponding to the ...
- 1,221
1
vote
0
answers
67
views
Maximum likelihood for Markov chain with missing observations
Let $\{X_i\}_{i=1}^{n}$ be the path of a Markov chain with 3 possible states $\{1,2,3\}$. Given the path, I know how to get the maximum likelihood estimator for the $3 \times 3$ transition probability ...
- 13
1
vote
0
answers
69
views
HMM - Does Foward-Backward algorithm has the same result as Viterbi if all transitions are possible?
I am attending a Bioinformatics class and we are learning about HMMs to make inference about DNA sequences.
Well, we recently learned about the forward-backward algorithm that gives us the ...
- 149
1
vote
0
answers
34
views
Face Recognition using HMM
I had learnt some of the research papers of Face Recognition using Hidden Markov Model.
Can you help me how Hidden Markov model is applied to face recognition?Also can you please give some numerical ...
1
vote
0
answers
33
views
Hidden Markov Model - Automatic Hidden State Interpretation
I am using a Hidden Markov Model to classify market regimes. For example, I train it on some asset returns and I get bullish and bearish regimes (2 hidden states). Visually inspecting the results ...
- 11
2
votes
1
answer
239
views
mean hitting time for $M/M/1$
Consider a manufacturing process with batches of raw materials
coming in. Suppose that the interarrival times of batches are i.i.d. exponential random variables
with rate λ and their processing times ...
- 21
1
vote
1
answer
218
views
Advantage of using Hidden Markov Model over Markov Chain
There are many problems that can be modeled using both Markov chain and Hidden Markov model (HMM). Can anyone please explain mathematically, why HMM should be preferred over Markov chain? Also, please ...
- 583