# Questions tagged [hidden-markov-models]

This tag is for questions relating to "Hidden Markov model", a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobservable (i.e. hidden) states.

80 questions
Filter by
Sorted by
Tagged with
19 views

### Emulating any Markov Process of N states using a restricted Markov Process of more than N states

Let's define an unrestricted family of Markov Processes on $N$ states as the set of all possible Markov Processes using the states $1, 2, ... N$. To clarify, if we let $A$ be the current state and $B$ ...
12 views

### Is there any references for solving inverse Ising problem w.r.t. some objective functions other than MaxLikelihood

I am trying to formulate an inverse Ising problem that optimizes some defined objective functions other than maximum likelihood. I am pretty new to this field (only some background on Markov random ...
1 vote
15 views

• 241
52 views

145 views

### Probability of observing sequence Markov model

I have been trying to understand hidden Markov models with observational probability but I often find myself confused. I have discussed with my tutor for further help however, he is often rude and ...
• 147
1 vote
23 views

### A question on Cox-Processes (Markov-Modulated Poisson processes)

I´m trying to prove that a Markov-Modulated Poisson process could be seen as a 2-dimensional continuous time markov chain. For this, I'm considering a two state markov chain $\{J(t) : t \geq 0\}$ with ...
25 views

### a question about the direction of a flow of numbers

I am trying to figure out if a stream of numbers i say more positive or more negative or neutral. Let's say I have a stream of numbers that I can sample. I would like to estimate are these last x ...
• 1
1 vote
42 views

### On the equivalence of different assumptions of Hidden Markov Models (HMM)

I am currently studying Hidden Markov Models (HMM). We denote the hidden quantities as $(X_0, \dots, X_n) \in \mathcal{X}^{n+1}$ and the observed quantities $(Y_1, \dots, Y_n) \in \mathcal{Y}^n$. The ...
• 45
22 views

### POMDPs: How to update transition probabilities after receiving new information

I am trying to model a POMDP based on user feedback dialogue and an observation set based on eye gaze. The goal is to specify which item the robot should pick up according which item the human's eye ...
• 23
1 vote
59 views

### Shortest Path using Markov Decision

For the following question, one solves the problem to find the shortest path from the Start to the End in the following maze using the Markov decision process. We formulate this problem as the ...
• 915
61 views

• 464
163 views

### Hidden Markov Model - why backward probability is conditional on the current state

I'm trying to understand hidden Markov model (HMM). Here is the material which I studied. It states that there are two assumptions in HMM (page 3): $P( q_i | q_1, ..., q_{i-1} ) = P( q_i | q_{i-1} )$ ...
• 172
1 vote
86 views

### Hidden Markov model - Probability of arbitrary (start and length) state sequence given all observations?

Consider a hidden Markov model $\lambda=\{A,B,q\}$, where $q$ is the initial probability matrix, $A$ the transition probability matrix, and $B$ the output probability distributions. Assume that we ...
• 175
38 views

### Bibliography about Phylogenetic trees from a math point of view

I am learning about phylogenetic trees, but it is difficult for me to find some documents/books focused on the maths. I saw this article https://www.researchgate.net/publication/...
• 1
42 views

### What's the application of doubly-stochastic matrices in engineering?

Today I learned the existence of such matrix . wolframe This is indeed a very interesting thing. I am wondering if there is any realworld application of such matrix. It seems that this matrix has ...
• 193
15 views

### How should forward-backward algorithm be implemented for a text file with multiple twitter posts?

Assuming i have a textfile which shows multiple twitter posts made by different users, how do we approach the forward-backward algorithm? Do we apply throughout the entire textfile by treating it as ...
1 vote
44 views

### Expectations in Hidden Markov Models

Background: Suppose I have an HMM characterized by the parameters $\theta = (\mathbf{A}, \mathbf{B}, \mathbf{\pi})$, where $A,B,\pi$ are the hidden state transition probability matrix, the emission ...
• 1,039
113 views

### Markov Chain: Conditional distribution at time $t$, given $t-1$ and $t+1$

For a Markov process given by $$x_t = \mu +\kappa(x_{t-1} - \mu) + \sigma \cdot \varepsilon_t$$ where $\varepsilon_t \sim N(0,1)$ and $\mu$, $\kappa$, $\sigma^2$ are the parameters, how would I find ...
1 vote
46 views

### Performing Inference on Hidden Markov Models with GMM Emission Probabilities

So I have a hidden markov model with two hidden states $z = a$ and $z = b$. My emission probabilities are given by:  P\left( x_{n} \mid z = a \right) = \frac{\pi_{1}}{\pi_{1} + \pi_{2}} \mathcal{N}(...
• 1,039
62 views

### Transformations of stochastic matrix that preserve equilibrium

I have a stochastic (Markov) matrix $W$. I would like to modify it, such that $W_{i,i}$ increases for all $i$ (and thus other elements decrease). However, I don't want to change the equilibrium ...
• 113
1 vote
42 views

### Measurability question about measure dependent on variables

Suppose $(Y, {\cal T}, \nu)$ is a measure space and $(X, {\cal S}, \mu_y)$ is a measure space for each fixed $y \in Y$; $\mu_y$ depends on $y$. What can we say about the ${\cal T}-{\cal B}$ ...
195 views

### Hidden Markov Model - understanding Viterbi algorithm

I try to understand the Viterbi algorithm for solving hidden Markov models. There is a pseudo-code of it in Wikipedia: In the row that marked in blue (starts with $T_2$) I don't understand: how does ...
• 338
23 views

### HMMs: Difference between the joint and conditional probabilities

I am having trouble in giving meaning to the joint and conditional probabilities related to the observations and states of HMMs in the Appendix A of Speech and Language Processing by Jurafsky and ...
• 1
76 views

### Unsure how to solve first-order Markov Chain problem

I am working on solving the following problem (I am new to Markov Chains): -It can be either rainy or sunny on a given day. -The probability that a rainy day is followed by a rainy day is 0.5. -The ...
1 vote
23 views

### Limiting probabilities in Markov Decision Process

Do limiting probabilities exist for discrete-time Markov Decision Process, given that the actions are deterministic? If so, how should it be calculated? Please provide links to notes/references, if ...
1 vote
39 views

### Best time to make a guess in an HMM with discounting?

Consider a hidden Markov model (HMM) with $2$ states and $2$ outputs. The transition probabilities are $p_{ij}$, $i,j=1,2$ and output (emission) probabilities are $q_{ik}$, $i=1,2$, $k=1,2$. Assume an ...
• 383
78 views

### Three state Markov chain

If I have Transition matrix $T=\begin{pmatrix} 2/3 &0 &1/3 \\ 1/4 &3/4 &0 \\ 1/3& 0 &2/3 \end{pmatrix}$ How would I get the quantity $R_i^{(n)}=\sum_{k=1}^{n}(T^k)_{ii}$...
1 vote
27 views

### How to determine optimal state sequence in HMM?

There are several criteria to determine state sequences in HMM. For example, most possible state for each individual observation, most possible pair states, and most possible sequence. Which one ...
• 123
1 vote
30 views

### partially observed hidden markov model

I am working on a data learning problem. Here's the framework: the data X_it for each observation i=1,...,N, time t=1,...,T, measures a bio-marker over time for an observation (continuous values). An ...
219 views

### How to interpret clusters on Markov chain time characteristics?

I have a complex network $G=(V,E)$ from multivariate financial time series in which a single vertex $v_i$ represents the types of states corresponding to the ...
• 1,221
1 vote
67 views

### Maximum likelihood for Markov chain with missing observations

Let $\{X_i\}_{i=1}^{n}$ be the path of a Markov chain with 3 possible states $\{1,2,3\}$. Given the path, I know how to get the maximum likelihood estimator for the $3 \times 3$ transition probability ...
• 13
1 vote
69 views

### HMM - Does Foward-Backward algorithm has the same result as Viterbi if all transitions are possible?

I am attending a Bioinformatics class and we are learning about HMMs to make inference about DNA sequences. Well, we recently learned about the forward-backward algorithm that gives us the ...
1 vote
34 views

### Face Recognition using HMM

I had learnt some of the research papers of Face Recognition using Hidden Markov Model. Can you help me how Hidden Markov model is applied to face recognition?Also can you please give some numerical ...
1 vote
33 views

### Hidden Markov Model - Automatic Hidden State Interpretation

I am using a Hidden Markov Model to classify market regimes. For example, I train it on some asset returns and I get bullish and bearish regimes (2 hidden states). Visually inspecting the results ...
• 11
239 views

### mean hitting time for $M/M/1$

Consider a manufacturing process with batches of raw materials coming in. Suppose that the interarrival times of batches are i.i.d. exponential random variables with rate λ and their processing times ...
• 21
1 vote