Skip to main content

Questions tagged [log-likelihood]

For questions that use the natural logarithm of a likelihood function.

Filter by
Sorted by
Tagged with
0 votes
0 answers
49 views

Finding the Cramer Rao bound

Let $x=(x_1,\dots,x_n)$ be a sample of i.i.d random variables with pdf $$f(x;\theta)=(1-\theta)\chi_{[-1/2,0]}+(1+\theta)\chi_{[0,1/2]}$$ where $\theta\in(-1,1)$. Find the Cramer Rao bound. So to do ...
Tutusaus's user avatar
  • 657
2 votes
2 answers
128 views

Computing the likelihood function

Let $x=(x_1,\dots,x_n)$ be a sample from $X_1,\dots,X_n$ independent identically distributed random variables with pdf $$f(x)=(1-\theta)1_{[-\frac{1}{2},0]}(x)+(1+\theta)1_{(0,\frac{1}{2}]}(x)$$ where ...
Tutusaus's user avatar
  • 657
0 votes
0 answers
18 views

What is the meaning of $p_{g_i}$ in this equation?

I am reading "The element of statistical learning" and having some question regarding equation 2.36. The book stated that: "A more interesting example is the multinomial likelihood for ...
alksdhalksjdb's user avatar
1 vote
0 answers
29 views

MLE of the covariance of the probabilistic PCA (multivariate normal distribution)

To get to the MLE of probalistic PCE, we start with the marginal distribution, which is defined with $x \sim \mathcal N(b, C)$ with $C=WW^\top + \sigma^2 I_d \text{ and } b \in \mathbb{R}^d, W \in \...
Lopsio's user avatar
  • 85
0 votes
0 answers
20 views

Why is maximizing the Cox partial likelihood meaningful

So the likelihood of seeing our data in survival analysis is (assuming simple case of no censorship and no ties): ...
user23773995's user avatar
-1 votes
1 answer
45 views

What distribution does the likelihood follow? [closed]

Consider a standard Gaussian distribution ($\mu=0$, $\sigma=1$), from which you sample $N=1000$ times. If you were to obtain the likelihood of the distribution parameters mu and sigma for each of the ...
Lucas's user avatar
  • 7
1 vote
1 answer
54 views

How to visualize conditional maximum likelihood estimation?

In Probabilistic Machine Learning (Murphy, 2022, p. 8) I'm stuck in this part: 1.2.1.6 Maximum likelihood estimation When fitting probabilistic models, it is common to use the negative log ...
filip augusto's user avatar
0 votes
0 answers
19 views

Pareto MLE with restricted range

The $\text{Pareto}(1, \alpha)$ MLE is $\frac{n}{\sum_{i = 1}^{n} \log x_{i}}$, where $X_{1}, X_{2}, ..., X_{n}$ are i.i.d. with $f(x | \alpha) = \alpha x^{-\alpha - 1}$. Further, add the constraint ...
narwahl's user avatar
  • 13
0 votes
0 answers
6 views

Shouldn't the product of importance sampling weight and likelihood equal 1 in PSIS?

I'm reading the 'Overthinking: Pareto-smoothed cross-validation' in Chapter 7.4 of Richard McElreath's textbook Statistical Rethinking 2nd edition. The author said: Cross-validation estimates the ...
EndlessHomework's user avatar
0 votes
2 answers
179 views

Deriving the Fisher information matrix for a reparameterised gamma distribution

Let $X \sim \mathrm{Gamma}(\alpha, \theta),$ where $$f(x) = \frac {x^{\alpha - 1} e^{-\frac x \theta}} {\theta^{\alpha}\Gamma(\alpha)}.$$ The log-likelihood function can be shown to be $$l(\alpha, \...
Ethan Mark's user avatar
  • 2,187
0 votes
1 answer
52 views

Likelihood computation for hidden markov models.

If we have a $2$-state model (i.e. the simplest non-trivial example) in a hidden markov model, and some generated observation-data $\mathcal{O}$ from the algorithm for generating observations. Is it ...
Ben123's user avatar
  • 1,268
0 votes
1 answer
80 views

Why do we use softmax with log-likelihood in deep learning?

Yes, as we use to say, it would be convenient when we initiate the back-propagation process because of the following formula: $$\nabla_x \ell_\text{LL} (S(x),\, e_k) = S(x) - e_k, \quad x \in \mathbb ...
Springfield's user avatar
3 votes
1 answer
58 views

Find conditional MLE of AR time series

I was given a model $r_{t} = ϕ_{0} + ϕ_{2}r_{t-2} + ϵ_{t}$ with $\epsilon_t \sim N(0,\sigma^2)$ and have to derive the likelihood of $(r_{3}, r_{4}, . . . , r_{T})$ conditional on $(r_{1}, r_{2})$ and ...
Tin Dao's user avatar
  • 31
2 votes
1 answer
514 views

Fisher Information Matrix for Weibull Distribution...

I wish to find the Fisher Information Matrix for the Weibull Distribution... I face two difficulties, I can't find any sufficient guide in internet to lead me to derive the Fisher Information Matrix.....
Gambit's user avatar
  • 285
0 votes
0 answers
48 views

Loss Equation for Training DPMs vs DDPMs

I'm currently trying to wrap my head around the training loss functions for DPMs and how they vary from DDPMs, however there are differences in how the papers describe the processes, making it ...
Tomas Premoli Muniagurria's user avatar
1 vote
1 answer
58 views

likelihood and fisher information of two dependent variables

Let $\{X, Y\}$ be two sets of variables that depend on the parameter $\theta$. Let $z_1 = f_1(X,Y), z_2 = f_2(X,Y)$ be two variables constructed from $\{X, Y\}$. The functions $f_i$'s are known. $I_{...
lost-neutrino's user avatar
0 votes
0 answers
24 views

Overall minimizer of the minus log-likelihood of joint Gaussian distribution

Consider the following model. There is a DAG $D_0$ whose p nodes correspond to random variables $X_1,...,X_p$: assume that $$X_1,...,X_p \sim N_p (0, \Sigma_0) \text{ with density } f_{ \Sigma_0} (\...
Jackie's user avatar
  • 553
0 votes
0 answers
75 views

Maximizing the likelihood over the truncated support always leads to strictly greater probability on the truncated region than original pdf?

Suppose $\mathbf{X}$ is a random variable with a finite support $\Omega$ and with some pdf $f(\cdot; \mathbf{v}_0)$ where $\mathbf{v}_0$ is the parameter. Define, $\mathcal{A}:= \{\mathbf{x}:S(\mathbf{...
entropy's user avatar
  • 147
2 votes
1 answer
681 views

Is the maximum likelihood estimator for the mean equal to the sample mean under all densities?

Suppose, I have a sample from an unknown distribution. I want to prove/disprove (mathematically!) the following statement: The maximum likelihood (ML) estimator for the (unknown) population mean will ...
entropy's user avatar
  • 147
1 vote
1 answer
42 views

Why do we write Bernoulli likelihood in terms of powers

Why do we write Bernoulli likelihood as follow: $L(p) = \prod_{i=1}^n p^{x_i}(1-p)^{(1-x_i)}$ and not like this: $L(p) = \prod_{i=1}^n [px_i +(1-p)(1-x_i)]$ Both expressions are equivalent. Is it only ...
Frostic's user avatar
  • 1,282
0 votes
1 answer
88 views

How to calculate an integral with an unknown number of integration variables?

How to calculate the following integral, which has an unknown number of integration variables? $$ \int\limits_{-\infty}^{+\infty}\cdots\int\limits_{-\infty}^{+\infty}\exp\left[-\dfrac{1}{2\theta}\sum_{...
woody's user avatar
  • 87
0 votes
1 answer
99 views

Maximum likehood estimate

I have a question regarding the correction of my exercise: Exercise 6. Let $Y_1,\dots,Y_n$ be i.i.d. such that $Y_i$ equals $1$ with probability $p$ and $-1$ with probability $1-p$, for all $i\in[n]$....
rogerhenry's user avatar
0 votes
0 answers
40 views

What is the log-likelihood formulation for a combined normally distributed dataset?

Consider a sample (i.i.d) $C_{1},...,C_{n}$ that is normally distributed ~$N(µ,1)$ and a sample (i.i.d) $D_{1},...,D_{n}$ ~$N(\theta,1)$, we say $µ\neq\theta$. Now I want to formulate a log-likelhood ...
DrDrunkenstein's user avatar
2 votes
1 answer
172 views

Function proportional to the log likelihood for the Gaussian distribution

The following question has been crossposted to CrossValidated upon recommendation from the community and a lack of responses here. Consider the following problem from a course on statistical ...
FD_bfa's user avatar
  • 4,343
1 vote
1 answer
75 views

Does almost sure convergence of $\frac{1}{m}L_m$ imply almost sure convergence of $\frac{1}{m}\max\limits_{k\leq m}L_k$?

Assume we have a distribution function which is controlled by a parameter $f(x;\theta)$. If we sample $m$ i.i.d samples from this distribution We can define the sum log-likelihood ratio of the $m$ ...
IdanC1s2's user avatar
1 vote
1 answer
130 views

MLE of Erlang Distribution

For sample $x_1,...,x_n$~Erlang, $f(x_i|k,\lambda)=\frac{\lambda^kx^{k-1}e^{-\lambda x_i}}{(k-1)!}$ Population has expectation $k/\lambda$ and variance $k/\lambda^2$. You may assume second derivative ...
Simon's user avatar
  • 21
1 vote
2 answers
346 views

How can I prove the Hessian of the log likelihood of the Generalized Error Distribution is negative definite?

I'm working with the multivariate generalized error distribution to model some data. (The parameterization that I am working with follows Graham Giller's work here: https://www.researchgate.net/...
Taylor's user avatar
  • 11
0 votes
1 answer
30 views

Likelihood equation of 2 samples geometric distribution.

Context Given there are 2 groups that can be modelled as a geometric distribution as follows: \begin{align*} f(x_i;p_1) &= p_1(1-p_1)^{x_i - 1} \; x_i = 1,2,... \; 0<p_1 <1 \\ f(y_i;p_2) &...
DanielMariam's user avatar
1 vote
1 answer
104 views

Using a Bin$(k,\theta)$ random sample ($k$ known), determine the posterior distribution for $\theta$ using a Jeffrey's prior.

My question here is, am I misinterpreting a binomial random sample? Doing some research on this I'm thinking perhaps I have misunderstood something - however my answer "feels" okay ...
spooleey's user avatar
  • 456
0 votes
1 answer
52 views

Fisher information $I(\sigma^2)$ and Cramer-Rao of estimator $S^2$

Let $X_1,...,X_n$ be a random sample with $X_1\sim N(\mu,\sigma^2)$. Compute $I(\sigma^2)$ and verify the Cramer-Rao theorem for the estimator $S^2$ Since $f_{\mu, \sigma^2} = \frac{1}{\sqrt{2\pi} \...
Quotenbanane's user avatar
  • 1,604
2 votes
1 answer
366 views

Multinomial Logistic Regression likelihood

Suppose we have the following parametric model for logistic regression: $$\phi_{i} = \frac{\exp{(a^{T}x_{[i]}})}{\sum_{k = 1} ^{M} \exp{(a^{T}x_{[k]})}}$$for $i = 1, \dots, M$ and that the parameter ...
Iamtrying's user avatar
  • 167
0 votes
1 answer
95 views

Partial derivative of a Likelihood function

I'm trying to get the partial derivatives $\frac{\partial L}{\partial w}$ of a log-Likelihood function $$ L(w) = \sum_{n=1}^{N}\sum_{k=1}^{K}y_{nk}\cdot log(\frac{e^{\sum_{i=1}^{D}w_{ki}x_{i}}}{\sum_{...
pietpaff's user avatar
3 votes
0 answers
16 views

Family of transformations can I find a density function?

Let's consider the family of transformations given by $$g_a(Y)=\begin{cases} \frac{e^{aY}-1}{a} & \text{ for } a\neq 0 \\ Y & \text{ for } a=0 \end{cases}$$ for $Y\in\mathbb{R}$. Analogous to ...
NabbKitha's user avatar
  • 540
1 vote
1 answer
212 views

MLE for a Borel distribution

I would appreciate help with on how to find the estimation $\hat{\beta}_{ML}$ for a Borel distribution. I am doing something wrong in, my guess, the likelihood function and therefore the final answer ...
kabin's user avatar
  • 391
0 votes
0 answers
36 views

how to simplify a log likelihood function

How to simplify/ get the log likehood of this? I understand basically the idea is to get the LN of both sides and simplify??
SOI's user avatar
  • 101
1 vote
0 answers
36 views

Show that a constrained probit likelihood is globally concave

I have a constrained probit likelihood and I would like to know if it is globally concave. My problem can be simplified as follows. Let $n\in\mathbb{N}^{\ast}$. Let $F$ be a function defined on $\...
Ari.stat's user avatar
  • 473
0 votes
0 answers
20 views

Where does discrete probabilities in Forward-Backward algorithm for Hidden Markov Models come from?

I am trying to derive a Forward-Backward algorithm used in Hidden Markov Models to compute the likelihood $P(x | \theta)$ that sample $x = (x_1, ... x_n)$ comes from HMM defined by set of parameters $\...
Brzoskwinia's user avatar
3 votes
1 answer
255 views

Quadratic Approximation for Log-Likelihood Ratio Processes

I'm trying to understand why the quadratic equation can approximate the log likelihood ratio, and how it is derived: $$\mathrm{Log}(\mathrm{LR})=\frac{1}{2}\left(\frac{\mathrm{MLE}-\theta}{S}\right)^2$...
Ela's user avatar
  • 33
1 vote
0 answers
36 views

Chaining a conditional likelihood function

Before I proceed, allow me to apologise in advance for potential abuse of notations. Let $\{x_t:\Omega \to \mathbb{R}\}_{t=1,\cdots,n}$ be a stochastic process, defined on a probability space $(\Omega,...
Carl's user avatar
  • 173
0 votes
1 answer
120 views

is the mix of convex and linear functions always convex function?

I want to prove that the following composed function $g \circ L$ is always (strictly) convex : \begin{alignat*}{3} &g&&(t&&) && =-\log(1-e^{-t}) \qquad && \text{...
Artashes's user avatar
  • 133
1 vote
0 answers
28 views

Differentiation of logLikelihood MLE

Given a log likelihood function, $$l=\sum_{i=1}^m\log[\frac{n!}{X_{i,1}!X_{i,2}!(n-X_{i,1}-X_{i,2})!}p_1^{X_{i,1}}p_2^{X_{i,2}}(1-p_1-p_2)^{n-X_{i,1}-X_{i,2}}]$$ To derive: with respect to $p_1$, $\...
darika's user avatar
  • 21
0 votes
1 answer
119 views

How to find the value that give the global maximum of a function?

I have got the following function: $$L(x,y) = \frac{(16.2x +0.9y+5.2)^{24} e^{-(16.2x +0.9y+5.2)}}{24!} \cdot \frac{(2.1x +4.2y+0.9)^8 e^{-(2.1x +4.2y+0.9)}}{8!} \tag{1}$$ and I am attempting at ...
gfgc's user avatar
  • 45
0 votes
1 answer
37 views

Simplifying $-\log\left(\left( \frac{1}{2\pi\sigma^2}\right)^{n/2} \exp \left( -\frac{1}{2\sigma^2} \sum_{i=1}^{n}(\mu-y_i)^2\right)\right)$

I'm not the most mathematically minded, but I'm doing my best to learn MLEs. We were provided the following original likelihood function of a Normal Distribution and I am having trouble understanding ...
guessing_numbers's user avatar
0 votes
1 answer
2k views

Derivate of the the negative log likelihood with composition

We want to solve the classification task, i.e., learn the parameters $\theta = (\mathbf{W}, \mathbf{b}) \in \mathbb{R}^{P\times K}\times \mathbb{R}^{K}$ of the function $f_\theta: \mathbb{R}^P \to [0, ...
CechMS's user avatar
  • 221
0 votes
0 answers
64 views

Likelihood-function : How to simplify it?

The fracture strength of hard bricks satisfies an Erlang distribution of order $2$. There are $n \in \mathbb{N}$ fracture strength tests that are carried out. We consider the statistical product model ...
Mary Star's user avatar
  • 14k
1 vote
1 answer
168 views

Show that expression is Gamma distribution with given parameters

I had the expression $$\frac{1}{p(y)}(n\lambda)^k e^{-n \lambda}$$ for $k \in \{0,1,2,...\}$ where we here have that $k=\sum_{i=1}^n y_i$ where I think I can ignore $p(y)$, but if not it is given by $...
Lifeni's user avatar
  • 558
1 vote
1 answer
312 views

Negative Log likelihood and Derivative of Gaussian Naive Bayes

I am trying to derive negative log likelihood of Gaussian Naive Bayes classifier and the derivatives of the parameters. So there are class labels $y \in {1, ..., k}$, and real valued vector of $d$ ...
DHH's user avatar
  • 57
1 vote
3 answers
63 views

All solutions to $f(a,b)+f(b,a)=0$

I am looking for all functions $f:(0,1)^2\to\mathbb{R}$ that satisfy $f(a,b)+f(b,a)=0$ for every $(a,b)\in(0,1)^2$. I know that $f\equiv 0$, $f(a,b)=c(a-b)$, and $f(a,b)=\pm\log(a/b)$ are such ...
user108's user avatar
  • 55
1 vote
1 answer
98 views

Likelihood of p and q given Y

Suppose that Jon has a coin with a probability of landing on heads p and Ann has a coin with a probability of q landing on heads. Each time they flip a coin and they write the sum of the outcome ...
Elmir's user avatar
  • 11
1 vote
2 answers
749 views

MLE + Cramér-Rao bound of a discrete random variable

Can someone explain how to compute the maximum likelihood estimator, the Fisher information and the Cramér-Rao bound of a discrete random variable please? I came across this exercise while reviewing ...
weldon's user avatar
  • 25

1
2 3 4 5