Questions tagged [hidden-markov-models]

The tag has no usage guidance.

0
votes
0answers
12 views

Bayes' Theorem application to Hidden Markov Model

So essentially I'm reading through my notes to try and understand everything and part of the working out on one of the questions was this. I understand that the first part is simply just Bayes theorem ...
0
votes
0answers
11 views

Errors in user-made HMM, Hidden Markov Model in MATLAB

This is my code from scratch. It is an attempt to make an HMM, but there must be serious errors in my logic and code. I need to observe the sequence of states and the sequence of observed, but I ...
0
votes
0answers
12 views

Transition density of hidden Markov model

Background info: We have a target moving in $\mathcal R^2$ according to the model; $$X_{n+1}=\Phi X_n+\Psi_z Z_{n}+\Psi_w W_{n+1}, \quad n \in \mathcal{N}$$ The information contained in $X_n=\{X_n^1,...
1
vote
1answer
60 views

Advantage of using Hidden Markov Model over Markov Chain

There are many problems that can be modeled using both Markov chain and Hidden Markov model (HMM). Can anyone please explain mathematically, why HMM should be preferred over Markov chain? Also, please ...
1
vote
0answers
32 views

Markov Assumption, Independence, and Conditional Probability Rules

In general, if $X,Y$ are conditionally independent on $Z$, then it follows that $$P(X,Y|Z) = P(X|Z)P(Y|Z)$$. I am working on a problem where I use the Markov assumption $$P(X_t|X_{t-1}, U_t)$$ ...
0
votes
0answers
12 views

Continous Observation with HMM & Gaussian

I've read that for HMMs with a discrete output we assign an emission probability $b_j(o_t)$ of emitting symbol $o_t$ when in state $S_j$. What I don't understand is why, in the case of continous ...
1
vote
0answers
24 views

Name of this Markov model, and how to estimate?

I came across a paper by Rodda (2004), who simulates interest rates with a Markov sequence. To simulate changes in the interest rates, they used the historical transition probabilities. Their ...
0
votes
0answers
15 views

Meaning of π*A*B in a hidden Markov model

I study hidden Markov models. Please let me check that I understood this correctly: π is the initial probability vector where each element is the probability of the corresponding state in the ...
1
vote
1answer
39 views

Probability of observation sequence not knowing previous observations

I'm trying to study hidden Markov models by solving some exercises, and stumbled upon this one: Given HMM 2, what is $P(O_{100} = A, O_{101} = A, O_{102} = A)$? I think that this can be solved ...
1
vote
0answers
21 views

Finding a recursive formula for these probabilities in a semi-Markov chain

For a semi-Markov chain, let $i,j\in S$ where $S$ is the space of states. Let also $P=(p_{ij})_{i,j\in S}$ be the matrix of transition probabilities. We define the following probabilities: $e_{i,j}(n)=...
0
votes
0answers
18 views

What does it mean that “general states are not conserving probability so, it could be neither a quantum system, nor a Markov chain”?

I read from here Since $\hat{H}$ is Hermitian matrix, its eigenvalues are real numbers. Components with positive eigenvalues decrease to zero and negative eigenvalues blow up to infinity. The only ...
0
votes
0answers
41 views

Picking path at random in DAG graph with probability equals to path weight.

I'm refering to the following paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.329.3653&rep=rep1&type=pdf. There is a lemma states: Let $G$ be a directed acyclic graph with ...
0
votes
0answers
44 views

When is it necessary to solve Kolmogorov forward equations (KFE) for a Markov Chain?

Say I have a continuous time markov chain, time homogeneous $X$ with a few states (say, 2). I want to know the distribution of where $X$ is at time $t$, call it $\mu_t$, which will be a vector of 2 ...
0
votes
0answers
12 views

Derivation of the beta-pass algorithm

I'm having trouble following along the derivation of this algorithm: I can't see how we get from (2.38) to (2.39). Does it have to do with conditional independence? I.e that $P(A,B|C) = P(A|C) * P(B|...
0
votes
0answers
20 views

Terminology: Dynamic Bayesian network with hidden process

I came across a problem which can be modelled using a special type of dynamic Bayesian network. I'm looking for a name for this kind of network, but could not find anything so far. It resembles a "...
0
votes
0answers
18 views

Clustering using Hidden Markov Model

We are using Hidden Markov Model to cluster gene expression data where every observation has univariate gaussian emission based on latent state. We have chosen to have three latent states and mean and ...
1
vote
0answers
84 views

What is the significance of “removal_effects” in ChannelAttribution R package?

I am trying to build a Markov model using the ChannelAttribution package available in R. Sample code to run this model is - <...
1
vote
1answer
44 views

Markov switching model joint distribution

Under a hidden markov model (HMM) we know that \begin{align*} p(\epsilon_1,\ldots,\epsilon_N,\Delta_1,\ldots,\Delta_N)=&p(\epsilon_1,\ldots,\epsilon_N\mid\Delta_1,\ldots,\Delta_N)p(\Delta_1,\...
1
vote
0answers
68 views

how can i set this problem as a continuous markov chain? [closed]

I request your help in order to know, how can i configure this problem as a continuous markov chain, need to define the main variable, the states, transition rates, and the matrix. I thought that it ...
1
vote
1answer
49 views

Can someone help me with this problem on Markov Chain?

There is a system which follows the equation: $X(n+1) = X(n) - 1 + A(n)$, where $A(n)$ is a random variable taking values $0,1,2$ with probabilities $p_0$, $p_1$ and $p_2$, respectively. Now, $X(n) = ...
2
votes
0answers
145 views

Transition Kernel and Transition matrix

I know that this question might sound too vague, but what is the difference between a transition matrix and transition kernel. In my stochastic processes class we mentioned the transition matrix with ...
1
vote
0answers
67 views

HMM with hidden outcomes, knapsack problem

I have an HMM where the outcomes are not always visible. Suppose we want to know the probability of being in a certain state and observing certain outcome, given that we only know the sum of all past ...
0
votes
1answer
136 views

Proving HMM forward algorithm via induction instead of “trellis diagram reasoning”.

Here are two example papers that use the trellis reasoning approach, which doesn't really click with my brain: Rabiner Jurafsky & Martin In fact every HMM exposition I've come across seems to ...
0
votes
1answer
979 views

Expected value for Markov Chain

Let $(X_n)_{n\geq0}$ is a Markov chain with state space $I=\{0,1,2,3,4,5,6 \}$ and transission matrix with transition probabilities $P_{i,i+1}=0.5, P_{i,0}=0.5$ and $P_{6,6}=1$ where $I\in\{0,1,2,3,4,...
0
votes
1answer
567 views

Transition Probabilities in HMM

In the Wiki page on the Viterbi Algorithm (https://en.wikipedia.org/wiki/Viterbi_algorithm) there is an example of an HMM describing patients being in states "fever" or "healthy". What I wish to ...
1
vote
0answers
23 views

Research on HMMs that look at before and after (instead of just what comes next)

I'm looking at a fill-in-the-blanks type of problem and I have not found a lot on it, which is why I am asking. It may be that it exists (and it probably does), but it goes by a name I'm not familiar ...
3
votes
1answer
123 views

Understanding equation for estimating parameter for hmm with continuous observations?

I am currently trying to understand how parameter are being reestimate for hidden markov models (hmm), using EM. What i seem to have problems understanding what the symbol emission probability is ...
9
votes
2answers
147 views

Fun mathematical model to announce pregnancy to husband

My husband is a mathematical modeler and I would love to give him a model to announce to him that we're pregnant! He is a pharmacometrician and a neurologist and often uses R, PKPD, and Markov models, ...
0
votes
1answer
90 views

Is this the collision entropoy of a hidden markov chain?

I am interested in the collision entropy rate of a hidden Markov chain, and I wonder if my way of calculating it is correct and if it has been described before. Definition Consider a Markov chain ...
1
vote
0answers
32 views

Approaching multiple optimization problems

I am currently looking at solving an optimization problem of the form $min\ f(arg\max\limits_{k}\quad\max\limits_{\theta} g(k,\theta,t))$ I would really appreciate some guidance in algorithms to ...
3
votes
1answer
538 views

Process properties of the maximum of two independent linear Brownian motions

Consider two independent linear Brownian motions $B'=(B'_t)_{t\geqslant0}$ and $B''=(B''_t)_{t\geqslant0}$, starting from $B'_0=B''_0=0$, and the process $X=(X_t)_{t\geqslant0}$ defined by $$X_t=\max\{...
1
vote
1answer
210 views

Hidden Markov Model Probability

I'm trying to understand how to find the probability of a given HMM. Here's the question that I have: Find $p(acca|EE5I)$ I did: $$\begin{align}p(acca|EE5I) &= E(a)*E(c)*5(c)*I(a)\\ &= .25 *...
1
vote
0answers
78 views

Convergence of Limiting Matrix

If the matrix $A_n$ is a markov transition matrix. Literature define de limiting matrix as $$\lim_{N \rightarrow \infty} \frac{1}{N} \sum_{n=0}^{N-1} A_n $$ where $\{A_n, n= 0,1,2,... \}$ is a ...
1
vote
0answers
128 views

Entropy of a mealy machine

I have a finite state machine with binary input, and I map each state non-injectively to letters of an output alphabet, here also binary. Assuming the binary input is uniformly random, how much ...
1
vote
0answers
16 views

Belief nets weight update

My problem is described in 13th slide of the following: http://helper.ipam.ucla.edu/publications/gss2012/gss2012_10596.pdf The only difference in my problem is that I have a single latent/hidden ...
0
votes
2answers
149 views

Expectation Maximization derivation using Jensen's inequality

I'm reading Andrew NG paper on Expectation Maximization algorithm in HMM Paper and I'm struggling with one 'simple' derivation: There is using Jensen Inequality, but I can't link (2) and (3). ...
0
votes
1answer
25 views

Is it ok to work with densities rather than probabilities (in statistical models)?

This is a recurrent problem I'm having difficulties addressing. Suppose I have some trained and ready to go Gaussian mixture model with $k$ clusters $c_1 c_2 c_3... c_k$ of known means and variances. ...
-1
votes
1answer
38 views

How to solve using markov chain

Two cars $X$ and $Y$ are in a race. The position of car $X$ as a function of time is described by $x(t)$. The position of car $Y$ is described by $y(t)$. Initially $x(t),y(t)>0$. Whenever $x(t)=y(t)...
1
vote
0answers
249 views

Binary Hidden Markov Model

Consider a binary HMM with 2 observed variables $O_n \in \{0,1\} \; \forall n \in \mathbb{N}$. Suppose that the hidden Markov process $X_n$ is characterised by a known transition probability matrix ...
3
votes
0answers
587 views

Hidden Markov Model, transition probabilities which are modeled with an exponential distribution

I'm looking at implementing an algorithm described in a paper, but I'm having trouble understanding how the transition probabilities for a Hidden Markov Model are defined. In the first sections, I ...
2
votes
3answers
212 views

Has there been significant study of deterministic Hidden Markov Models?

By 'deterministic Hidden Markov Models', I mean HMMs in which all state transition probabilities and output probabilities = 1 or 0. Have models subject to this restriction received any significant ...