Questions tagged [log-likelihood]

For questions that use the natural logarithm of a likelihood function.

Filter by
Sorted by
Tagged with
0 votes
1 answer
40 views

Likelihood computation for hidden markov models.

If we have a $2$-state model (i.e. the simplest non-trivial example) in a hidden markov model, and some generated observation-data $\mathcal{O}$ from the algorithm for generating observations. Is it ...
Ben123's user avatar
  • 868
0 votes
1 answer
41 views

Why do we use softmax with log-likelihood in deep learning?

Yes, as we use to say, it would be convenient when we initiate the back-propagation process because of the following formula: $$\nabla_x \ell_\text{LL} (S(x),\, e_k) = S(x) - e_k, \quad x \in \mathbb ...
Springfield's user avatar
3 votes
1 answer
50 views

Find conditional MLE of AR time series

I was given a model $r_{t} = ϕ_{0} + ϕ_{2}r_{t-2} + ϵ_{t}$ with $\epsilon_t \sim N(0,\sigma^2)$ and have to derive the likelihood of $(r_{3}, r_{4}, . . . , r_{T})$ conditional on $(r_{1}, r_{2})$ and ...
Tin Dao's user avatar
  • 31
2 votes
1 answer
158 views

Fisher Information Matrix for Weibull Distribution...

I wish to find the Fisher Information Matrix for the Weibull Distribution... I face two difficulties, I can't find any sufficient guide in internet to lead me to derive the Fisher Information Matrix.....
Gambit's user avatar
  • 187
0 votes
0 answers
39 views

Loss Equation for Training DPMs vs DDPMs

I'm currently trying to wrap my head around the training loss functions for DPMs and how they vary from DDPMs, however there are differences in how the papers describe the processes, making it ...
Tomas Premoli Muniagurria's user avatar
1 vote
1 answer
43 views

likelihood and fisher information of two dependent variables

Let $\{X, Y\}$ be two sets of variables that depend on the parameter $\theta$. Let $z_1 = f_1(X,Y), z_2 = f_2(X,Y)$ be two variables constructed from $\{X, Y\}$. The functions $f_i$'s are known. $I_{...
lost-neutrino's user avatar
0 votes
0 answers
24 views

Overall minimizer of the minus log-likelihood of joint Gaussian distribution

Consider the following model. There is a DAG $D_0$ whose p nodes correspond to random variables $X_1,...,X_p$: assume that $$X_1,...,X_p \sim N_p (0, \Sigma_0) \text{ with density } f_{ \Sigma_0} (\...
Jackie's user avatar
  • 553
0 votes
0 answers
69 views

Maximizing the likelihood over the truncated support always leads to strictly greater probability on the truncated region than original pdf?

Suppose $\mathbf{X}$ is a random variable with a finite support $\Omega$ and with some pdf $f(\cdot; \mathbf{v}_0)$ where $\mathbf{v}_0$ is the parameter. Define, $\mathcal{A}:= \{\mathbf{x}:S(\mathbf{...
entropy's user avatar
  • 127
2 votes
1 answer
434 views

Is the maximum likelihood estimator for the mean equal to the sample mean under all densities?

Suppose, I have a sample from an unknown distribution. I want to prove/disprove (mathematically!) the following statement: The maximum likelihood (ML) estimator for the (unknown) population mean will ...
entropy's user avatar
  • 127
1 vote
1 answer
39 views

Why do we write Bernoulli likelihood in terms of powers

Why do we write Bernoulli likelihood as follow: $L(p) = \prod_{i=1}^n p^{x_i}(1-p)^{(1-x_i)}$ and not like this: $L(p) = \prod_{i=1}^n [px_i +(1-p)(1-x_i)]$ Both expressions are equivalent. Is it only ...
Frostic's user avatar
  • 1,262
0 votes
1 answer
83 views

How to calculate an integral with an unknown number of integration variables?

How to calculate the following integral, which has an unknown number of integration variables? $$ \int\limits_{-\infty}^{+\infty}\cdots\int\limits_{-\infty}^{+\infty}\exp\left[-\dfrac{1}{2\theta}\sum_{...
woody's user avatar
  • 87
0 votes
0 answers
30 views

log-likelihood: changing relative contribution of two summation terms

Suppose I have a log-likelihood of the form $$\mathcal{L} = \sum_{i = 1}^{n} a_i + \sum_{j = 1}^{m} b_j,$$ where $a_i$ and $b_j$ are some independent log-probabilities. The problem is, the second sum $...
Dr. Timofey Prodanov's user avatar
0 votes
1 answer
94 views

Maximum likehood estimate

I have a question regarding the correction of my exercise: Exercise 6. Let $Y_1,\dots,Y_n$ be i.i.d. such that $Y_i$ equals $1$ with probability $p$ and $-1$ with probability $1-p$, for all $i\in[n]$....
rogerhenry's user avatar
0 votes
0 answers
20 views

What is the log-likelihood formulation for a combined normally distributed dataset?

Consider a sample (i.i.d) $C_{1},...,C_{n}$ that is normally distributed ~$N(µ,1)$ and a sample (i.i.d) $D_{1},...,D_{n}$ ~$N(\theta,1)$, we say $µ\neq\theta$. Now I want to formulate a log-likelhood ...
DrDrunkenstein's user avatar
0 votes
0 answers
28 views

Maximum Likelihood Estimation with degenerate functions

I have data which can be described by a background term, $f(x)$, and a signal term, $g(x,\theta)$. In this particular case, $f(x) = x^a$ is a single power law whose index is known, and $g(x) = A_0 \...
Chris Kervick's user avatar
2 votes
1 answer
154 views

Function proportional to the log likelihood for the Gaussian distribution

The following question has been crossposted to CrossValidated upon recommendation from the community and a lack of responses here. Consider the following problem from a course on statistical ...
FD_bfa's user avatar
  • 3,853
1 vote
1 answer
64 views

Does almost sure convergence of $\frac{1}{m}L_m$ imply almost sure convergence of $\frac{1}{m}\max\limits_{k\leq m}L_k$?

Assume we have a distribution function which is controlled by a parameter $f(x;\theta)$. If we sample $m$ i.i.d samples from this distribution We can define the sum log-likelihood ratio of the $m$ ...
IdanC1s2's user avatar
0 votes
0 answers
39 views

How to rewrite this partial derivative of in terms of the parameter?

I'm trying to estimate the parameters of a Hawkes process using an EM algorithm. I found this paper where in the maximisation step, for parameter $\mu$, we have $$\mu^{k+1} = \frac{\sum^n_{i=1}p^k_{ii}...
Joe's user avatar
  • 599
1 vote
1 answer
96 views

MLE of Erlang Distribution

For sample $x_1,...,x_n$~Erlang, $f(x_i|k,\lambda)=\frac{\lambda^kx^{k-1}e^{-\lambda x_i}}{(k-1)!}$ Population has expectation $k/\lambda$ and variance $k/\lambda^2$. You may assume second derivative ...
Simon's user avatar
  • 21
1 vote
2 answers
293 views

How can I prove the Hessian of the log likelihood of the Generalized Error Distribution is negative definite?

I'm working with the multivariate generalized error distribution to model some data. (The parameterization that I am working with follows Graham Giller's work here: https://www.researchgate.net/...
Taylor's user avatar
  • 11
0 votes
1 answer
22 views

Likelihood equation of 2 samples geometric distribution.

Context Given there are 2 groups that can be modelled as a geometric distribution as follows: \begin{align*} f(x_i;p_1) &= p_1(1-p_1)^{x_i - 1} \; x_i = 1,2,... \; 0<p_1 <1 \\ f(y_i;p_2) &...
DanielMariam's user avatar
0 votes
0 answers
71 views

Derivatives and Integrals of the Likelihood Function

I am reading the following notes : https://www.nan-ye.com/teach/stat3500/slides/12.pdf (page 16) Here, it says that " The usual log-likelihood is an integral of the score function." I have ...
Uk rain troll's user avatar
1 vote
1 answer
71 views

Using a Bin$(k,\theta)$ random sample ($k$ known), determine the posterior distribution for $\theta$ using a Jeffrey's prior.

My question here is, am I misinterpreting a binomial random sample? Doing some research on this I'm thinking perhaps I have misunderstood something - however my answer "feels" okay ...
spooleey's user avatar
  • 478
0 votes
1 answer
48 views

Fisher information $I(\sigma^2)$ and Cramer-Rao of estimator $S^2$

Let $X_1,...,X_n$ be a random sample with $X_1\sim N(\mu,\sigma^2)$. Compute $I(\sigma^2)$ and verify the Cramer-Rao theorem for the estimator $S^2$ Since $f_{\mu, \sigma^2} = \frac{1}{\sqrt{2\pi} \...
Quotenbanane's user avatar
  • 1,584
2 votes
1 answer
317 views

Multinomial Logistic Regression likelihood

Suppose we have the following parametric model for logistic regression: $$\phi_{i} = \frac{\exp{(a^{T}x_{[i]}})}{\sum_{k = 1} ^{M} \exp{(a^{T}x_{[k]})}}$$for $i = 1, \dots, M$ and that the parameter ...
Iamtrying's user avatar
  • 167
0 votes
1 answer
83 views

Partial derivative of a Likelihood function

I'm trying to get the partial derivatives $\frac{\partial L}{\partial w}$ of a log-Likelihood function $$ L(w) = \sum_{n=1}^{N}\sum_{k=1}^{K}y_{nk}\cdot log(\frac{e^{\sum_{i=1}^{D}w_{ki}x_{i}}}{\sum_{...
pietpaff's user avatar
3 votes
0 answers
16 views

Family of transformations can I find a density function?

Let's consider the family of transformations given by $$g_a(Y)=\begin{cases} \frac{e^{aY}-1}{a} & \text{ for } a\neq 0 \\ Y & \text{ for } a=0 \end{cases}$$ for $Y\in\mathbb{R}$. Analogous to ...
NabbKitha's user avatar
  • 500
1 vote
1 answer
168 views

MLE for a Borel distribution

I would appreciate help with on how to find the estimation $\hat{\beta}_{ML}$ for a Borel distribution. I am doing something wrong in, my guess, the likelihood function and therefore the final answer ...
kabin's user avatar
  • 391
0 votes
0 answers
31 views

how to simplify a log likelihood function

How to simplify/ get the log likehood of this? I understand basically the idea is to get the LN of both sides and simplify??
SOI's user avatar
  • 101
1 vote
0 answers
32 views

Show that a constrained probit likelihood is globally concave

I have a constrained probit likelihood and I would like to know if it is globally concave. My problem can be simplified as follows. Let $n\in\mathbb{N}^{\ast}$. Let $F$ be a function defined on $\...
Ari.stat's user avatar
  • 453
0 votes
0 answers
20 views

Where does discrete probabilities in Forward-Backward algorithm for Hidden Markov Models come from?

I am trying to derive a Forward-Backward algorithm used in Hidden Markov Models to compute the likelihood $P(x | \theta)$ that sample $x = (x_1, ... x_n)$ comes from HMM defined by set of parameters $\...
Brzoskwinia's user avatar
3 votes
1 answer
192 views

Quadratic Approximation for Log-Likelihood Ratio Processes

I'm trying to understand why the quadratic equation can approximate the log likelihood ratio, and how it is derived: $$\mathrm{Log}(\mathrm{LR})=\frac{1}{2}\left(\frac{\mathrm{MLE}-\theta}{S}\right)^2$...
Ela's user avatar
  • 33
1 vote
0 answers
34 views

Chaining a conditional likelihood function

Before I proceed, allow me to apologise in advance for potential abuse of notations. Let $\{x_t:\Omega \to \mathbb{R}\}_{t=1,\cdots,n}$ be a stochastic process, defined on a probability space $(\Omega,...
Carl's user avatar
  • 173
0 votes
1 answer
104 views

is the mix of convex and linear functions always convex function?

I want to prove that the following composed function $g \circ L$ is always (strictly) convex : \begin{alignat*}{3} &g&&(t&&) && =-\log(1-e^{-t}) \qquad && \text{...
Artashes's user avatar
  • 123
1 vote
0 answers
28 views

Differentiation of logLikelihood MLE

Given a log likelihood function, $$l=\sum_{i=1}^m\log[\frac{n!}{X_{i,1}!X_{i,2}!(n-X_{i,1}-X_{i,2})!}p_1^{X_{i,1}}p_2^{X_{i,2}}(1-p_1-p_2)^{n-X_{i,1}-X_{i,2}}]$$ To derive: with respect to $p_1$, $\...
darika's user avatar
  • 21
0 votes
1 answer
113 views

How to find the value that give the global maximum of a function?

I have got the following function: $$L(x,y) = \frac{(16.2x +0.9y+5.2)^{24} e^{-(16.2x +0.9y+5.2)}}{24!} \cdot \frac{(2.1x +4.2y+0.9)^8 e^{-(2.1x +4.2y+0.9)}}{8!} \tag{1}$$ and I am attempting at ...
gfgc's user avatar
  • 45
0 votes
1 answer
35 views

Simplifying $-\log\left(\left( \frac{1}{2\pi\sigma^2}\right)^{n/2} \exp \left( -\frac{1}{2\sigma^2} \sum_{i=1}^{n}(\mu-y_i)^2\right)\right)$

I'm not the most mathematically minded, but I'm doing my best to learn MLEs. We were provided the following original likelihood function of a Normal Distribution and I am having trouble understanding ...
guessing_numbers's user avatar
0 votes
1 answer
2k views

Derivate of the the negative log likelihood with composition

We want to solve the classification task, i.e., learn the parameters $\theta = (\mathbf{W}, \mathbf{b}) \in \mathbb{R}^{P\times K}\times \mathbb{R}^{K}$ of the function $f_\theta: \mathbb{R}^P \to [0, ...
CechMS's user avatar
  • 221
0 votes
0 answers
55 views

Likelihood-function : How to simplify it?

The fracture strength of hard bricks satisfies an Erlang distribution of order $2$. There are $n \in \mathbb{N}$ fracture strength tests that are carried out. We consider the statistical product model ...
Mary Star's user avatar
  • 13.9k
1 vote
1 answer
155 views

Show that expression is Gamma distribution with given parameters

I had the expression $$\frac{1}{p(y)}(n\lambda)^k e^{-n \lambda}$$ for $k \in \{0,1,2,...\}$ where we here have that $k=\sum_{i=1}^n y_i$ where I think I can ignore $p(y)$, but if not it is given by $...
Lifeni's user avatar
  • 558
1 vote
1 answer
280 views

Negative Log likelihood and Derivative of Gaussian Naive Bayes

I am trying to derive negative log likelihood of Gaussian Naive Bayes classifier and the derivatives of the parameters. So there are class labels $y \in {1, ..., k}$, and real valued vector of $d$ ...
DHH's user avatar
  • 57
1 vote
3 answers
61 views

All solutions to $f(a,b)+f(b,a)=0$

I am looking for all functions $f:(0,1)^2\to\mathbb{R}$ that satisfy $f(a,b)+f(b,a)=0$ for every $(a,b)\in(0,1)^2$. I know that $f\equiv 0$, $f(a,b)=c(a-b)$, and $f(a,b)=\pm\log(a/b)$ are such ...
user108's user avatar
  • 55
1 vote
1 answer
97 views

Likelihood of p and q given Y

Suppose that Jon has a coin with a probability of landing on heads p and Ann has a coin with a probability of q landing on heads. Each time they flip a coin and they write the sum of the outcome ...
Elmir's user avatar
  • 11
1 vote
2 answers
649 views

MLE + Cramér-Rao bound of a discrete random variable

Can someone explain how to compute the maximum likelihood estimator, the Fisher information and the Cramér-Rao bound of a discrete random variable please? I came across this exercise while reviewing ...
weldon's user avatar
  • 15
0 votes
2 answers
83 views

Roots of log binomial likelihood equation

I have been wondering if there is a way to approximate the roots of the log binomial likelihood equation. To be clear the equation is $$a \cdot \log\left(x\right) + b \cdot \log\left(1 - x\right) = t$$...
user5659375's user avatar
0 votes
0 answers
18 views

Formulate a likelihood function for an uncertain discrete "n", where y is binom(n,p), and we know y and p

The problem goes as followed: multiple balls have been tossed into a box, but we do not know how many that were tossed. However, we know that 5 of the balls that were tossed had a blue colour ($y=5$), ...
Quantgrind's user avatar
3 votes
1 answer
543 views

When does sup and function commute?

For $f,g$ real-valued functions, $f$ weakly increasing and continuous, $A\subseteq \mathbb{R}$, can we say \begin{align*} \sup_{x\in A}f(g(x))=f(\sup_{x\in A}g(x)) \end{align*} I ask because I notice ...
Golden_Ratio's user avatar
  • 12.6k
1 vote
0 answers
42 views

Log likelihood from multi-variate Normal distribution?

If we consider a linear mixed model that consider fixed effect and random effect: $$y = X\beta + W_uu + e,$$ where $n$ is the sample size $y \in \mathbb{R}^{n \times 1}$ is the response vector $X \in ...
Yuki.F's user avatar
  • 751
1 vote
0 answers
116 views

Differentiating log likelihood with respect to data

I'm working with the probabilistic model derived from the KL NMF, in which we sample each entry of a data matrix X from the distribution $\forall_{i,j},X_{ij}\sim\text{Poisson}\left(\left[WH\right]_{...
Elaf's user avatar
  • 11
0 votes
0 answers
51 views

Log-Likelihood of Piecewise-Defined Function

This is a follow-up to this question, which might be too difficult. So I break the problem down. What I need to find is the log-likelihood of the following function: $$f(x, a_{sg}, a_{pl}, p) = \begin{...
hyhno01's user avatar
  • 13

1
2 3 4 5