Questions tagged [log-likelihood]

For questions that use the natural logarithm of a likelihood function.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
26 views

How do I find the expectation value of a log likelihood function?

I am working on a problem where I need to find the fisher scoring update for two maximum likelihood parameters $a_1$ and $a_2$ that are part of a Poisson distribution. I have the derivatives of the ...
  • 1
0 votes
0 answers
12 views

negative sign of the Log-likelihood gradient

I was watching an explanation about how to derivate the negative log-likelihood using gradient descent, Gradient Descent - THE MATH YOU SHOULD KNOW but at 8:27 says that as this is a loss function we ...
1 vote
0 answers
19 views

Show that a constrained probit likelihood is globally concave

I have a constrained probit likelihood and I would like to know if it is globally concave. My problem can be simplified as follows. Let $n\in\mathbb{N}^{\ast}$. Let $F$ be a function defined on $\...
  • 363
0 votes
0 answers
39 views

Calculate maximum likelihood of Ridge regression

I'm reading a bit about ridge regression, $Ridge= \sum_{i=1}^{n}\left(y_i-\beta_0-\sum_{j=1}^{p}\beta_jx_{ij}\right)^2+\lambda\sum_{j=1}^{p}\beta_j^2$ and I wanted to try and implement it. So I ...
  • 53
0 votes
0 answers
13 views

Where does discrete probabilities in Forward-Backward algorithm for Hidden Markov Models come from?

I am trying to derive a Forward-Backward algorithm used in Hidden Markov Models to compute the likelihood $P(x | \theta)$ that sample $x = (x_1, ... x_n)$ comes from HMM defined by set of parameters $\...
3 votes
1 answer
42 views

Quadratic Approximation for Log-Likelihood Ratio Processes

I'm trying to understand why the quadratic equation can approximate the log likelihood ratio, and how it is derived. enter image description here Is this approximated using Taylor's series or normal ...
0 votes
0 answers
52 views

Mixture of Markov Models question

This is a follow up from this question. Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
  • 29
1 vote
0 answers
20 views

Chaining a conditional likelihood function

Before I proceed, allow me to apologise in advance for potential abuse of notations. Let $\{x_t:\Omega \to \mathbb{R}\}_{t=1,\cdots,n}$ be a stochastic process, defined on a probability space $(\Omega,...
  • 151
0 votes
1 answer
40 views

is the mix of convex and linear functions always convex function?

I want to prove that the following composed function $g \circ L$ is always (strictly) convex : \begin{alignat*}{3} &g&&(t&&) && =-\log(1-e^{-t}) \qquad && \text{...
0 votes
0 answers
16 views

Negative log-likelihood of Gaussian distributions

In a paper about collective outlier detection, I found the following penalized cost formula. $$ \sum \limits_{t\notin \cup \left[{\tilde{s}}_i+1,{\tilde{e}}_i\right]}\mathcal{C}\left({\mathbf{x}}_t,{\...
  • 1,914
0 votes
0 answers
11 views

Does the negative log likelihood of Discrete Weibull have Lipschitz gradients?

I am working on different convergence analyses of optimization algorithms for machine learning. However, almost all of them are based on the assumption that the objective function $f(x)$ has Lipschitz ...
0 votes
0 answers
19 views

Beta binomial regression model: sufficient statistic of the regression coefficients

I have the following beta-binomial with logit link function model: $$f(y_i|\pi_i,\phi)\sim\operatorname{binomial}(p_i), \text{with }p_i \sim \operatorname{beta}(\frac{\pi_i }{\phi},\frac{(1-\pi_i)}{\...
0 votes
0 answers
17 views

Is there a probabilistic concept or theory for infinitesimal logarithmic product interpretation of integral?

If we have a number of independent events in probability, we can calculate it's likelihood : $$\prod_{\forall i} p_{i}$$ We can also consider ( where $H$ is the Heaviside step function ) $$\int L(t) H(...
  • 24.5k
0 votes
0 answers
12 views

MLE & MLP Comparison

I am studying the allele frequency in population genetics and I have some questions: When using a conjugate prior and comparing the MLE and MLP results for allele frequency, how does the mean squared ...
1 vote
0 answers
26 views

Differentiation of logLikelihood MLE

Given a log likelihood function, $$l=\sum_{i=1}^m\log[\frac{n!}{X_{i,1}!X_{i,2}!(n-X_{i,1}-X_{i,2})!}p_1^{X_{i,1}}p_2^{X_{i,2}}(1-p_1-p_2)^{n-X_{i,1}-X_{i,2}}]$$ To derive: with respect to $p_1$, $\...
  • 21
0 votes
0 answers
45 views

Cross entropy and log-likelihood

I think it's pretty clear to me that average log-likelihood is equivalent to negative cross-entropy for discrete distributions, as shown here: $$\frac{1}{N}\log\mathcal{L}(\theta) = \frac{1}{N}\log \...
0 votes
1 answer
58 views

How to find the value that give the global maximum of a function?

I have got the following function: $$L(x,y) = \frac{(16.2x +0.9y+5.2)^{24} e^{-(16.2x +0.9y+5.2)}}{24!} \cdot \frac{(2.1x +4.2y+0.9)^8 e^{-(2.1x +4.2y+0.9)}}{8!} \tag{1}$$ and I am attempting at ...
  • 45
0 votes
0 answers
27 views

How to find the global maximum of a likelihood-function?

I have an expression for a likelihood function,$L_{A,B}$, with two parameters, $(A,B)$. I want to find the global maximum for this function, how can this be done? A colleague has told me that: "...
0 votes
1 answer
27 views

Simplifying $-\log\left(\left( \frac{1}{2\pi\sigma^2}\right)^{n/2} \exp \left( -\frac{1}{2\sigma^2} \sum_{i=1}^{n}(\mu-y_i)^2\right)\right)$

I'm not the most mathematically minded, but I'm doing my best to learn MLEs. We were provided the following original likelihood function of a Normal Distribution and I am having trouble understanding ...
0 votes
0 answers
12 views

How to find the likelihood probability of an exponential data model

I have a very basic knowledge in statistics, so I am struggling a bit with the ideas of Bayesian inference. My data model looks like this, $$ z(t) = \sum_{n = 1}^{N} e^{j 4\pi/\lambda \sqrt{(x_{n, t -...
0 votes
0 answers
28 views

log-likelihood of a function

Hi I am trying to get the log-likelihood of the above function, I Know i get the likelihood by adding the Product Notation and then I need the log of that but my math kinda lets me down then, what ...
  • 33
0 votes
1 answer
71 views

Derivate of the the negative log likelihood with composition

We want to solve the classification task, i.e., learn the parameters $\theta = (\mathbf{W}, \mathbf{b}) \in \mathbb{R}^{P\times K}\times \mathbb{R}^{K}$ of the function $f_\theta: \mathbb{R}^P \to [0, ...
  • 201
0 votes
0 answers
62 views

Is mean of derivative equal to derivative of mean anf if yes, under which conditions

Coonsidering the Likelihoog to be Gaussian, I woul like to know if the mean of derivative is equal to the derivative of the mean ? Here is an illustration in the context. of the computation of a ...
0 votes
0 answers
31 views

Likelihood-function : How to simplify it?

The fracture strength of hard bricks satisfies an Erlang distribution of order $2$. There are $n \in \mathbb{N}$ fracture strength tests that are carried out. We consider the statistical product model ...
  • 12.6k
0 votes
0 answers
16 views

Simplification and implementation of 2D log likelihood Bernoulli model

I'm trying to implement a paper titled "The spatial scan statistic: A new method for spatial aggregation of categorical raster maps" that uses the "log form" of a likelihood ...
1 vote
1 answer
86 views

Show that expression is Gamma distribution with given parameters

I had the expression $$\frac{1}{p(y)}(n\lambda)^k e^{-n \lambda}$$ for $k \in \{0,1,2,...\}$ where we here have that $k=\sum_{i=1}^n y_i$ where I think I can ignore $p(y)$, but if not it is given by $...
  • 621
1 vote
1 answer
123 views

Negative Log likelihood and Derivative of Gaussian Naive Bayes

I am trying to derive negative log likelihood of Gaussian Naive Bayes classifier and the derivatives of the parameters. So there are class labels $y \in {1, ..., k}$, and real valued vector of $d$ ...
  • 57
0 votes
0 answers
14 views

Finding out the generative log-likelihood $\log(p(x|t))$

The question ask to build classifiers to label images of handwritten digits. Each image is $8$ by $8$ pixels and is represented as a vector of dimension $64$ by listing all the pixel values in raster ...
  • 603
0 votes
0 answers
24 views

Calculate likelihood odds ratio with t-score

I'm researching in NIFTY and got struck on the calculation of log likelihood ratio. I'm having $2$ t-score $t_1$ and $t_2$ generated from binary hypothesis. The first test has null hypothesis is $|...
1 vote
3 answers
57 views

All solutions to $f(a,b)+f(b,a)=0$

I am looking for all functions $f:(0,1)^2\to\mathbb{R}$ that satisfy $f(a,b)+f(b,a)=0$ for every $(a,b)\in(0,1)^2$. I know that $f\equiv 0$, $f(a,b)=c(a-b)$, and $f(a,b)=\pm\log(a/b)$ are such ...
  • 55
1 vote
1 answer
66 views

Likelihood of p and q given Y

Suppose that Jon has a coin with a probability of landing on heads p and Ann has a coin with a probability of q landing on heads. Each time they flip a coin and they write the sum of the outcome ...
  • 11
1 vote
2 answers
200 views

MLE + Cramér-Rao bound of a discrete random variable

Can someone explain how to compute the maximum likelihood estimator, the Fisher information and the Cramér-Rao bound of a discrete random variable please? I came across this exercise while reviewing ...
  • 15
0 votes
0 answers
32 views

Which optimization method should I use?

I am trying to maximize a likelihood function using the Newton-Raphson method; however, it seems that the Hessian matrix is badly scaled by definition of my optimization problem. Now, I do not ...
0 votes
2 answers
76 views

Roots of log binomial likelihood equation

I have been wondering if there is a way to approximate the roots of the log binomial likelihood equation. To be clear the equation is $$a \cdot \log\left(x\right) + b \cdot \log\left(1 - x\right) = t$$...
  • 3
0 votes
0 answers
17 views

Formulate a likelihood function for an uncertain discrete "n", where y is binom(n,p), and we know y and p

The problem goes as followed: multiple balls have been tossed into a box, but we do not know how many that were tossed. However, we know that 5 of the balls that were tossed had a blue colour ($y=5$), ...
2 votes
1 answer
151 views

When does sup and function commute?

For $f,g$ real-valued functions, $f$ weakly increasing and continuous, $A\subseteq \mathbb{R}$, can we say \begin{align*} \sup_{x\in A}f(g(x))=f(\sup_{x\in A}g(x)) \end{align*} I ask because I notice ...
  • 7,938
1 vote
0 answers
31 views

Log likelihood from multi-variate Normal distribution?

If we consider a linear mixed model that consider fixed effect and random effect: $$y = X\beta + W_uu + e,$$ where $n$ is the sample size $y \in \mathbb{R}^{n \times 1}$ is the response vector $X \in ...
  • 711
1 vote
0 answers
67 views

Differentiating log likelihood with respect to data

I'm working with the probabilistic model derived from the KL NMF, in which we sample each entry of a data matrix X from the distribution $\forall_{i,j},X_{ij}\sim\text{Poisson}\left(\left[WH\right]_{...
  • 11
0 votes
0 answers
38 views

Log-Likelihood of Piecewise-Defined Function

This is a follow-up to this question, which might be too difficult. So I break the problem down. What I need to find is the log-likelihood of the following function: $$f(x, a_{sg}, a_{pl}, p) = \begin{...
  • 13
1 vote
1 answer
90 views

Log-Likelihood Function of Piecewise-Defined Function

I'm a data analyst, no mathematician. Mostly I do 'standard' stuff such as linear mixed-effects regressions or generalized additive mixed models. But now I need to determine the maximum log-likelihood ...
  • 13
1 vote
1 answer
87 views

Likelihood ratio hypothesis test for the exponential distribution.

I am trying to understand the following logic that I found on the internet that implements a hypothesis test on the exponential distribution using the likelihood ratio test. We want to test the ...
1 vote
1 answer
38 views

How to find the log-likelihood for this density?

We let $(X,Y)$ be stochastic variables with values in $N × R$ determined by $ X = x $ being subtracted from one Poisson distribution with mean $\lambda$ after which Y is subtracted from a gamma ...
  • 621
1 vote
1 answer
100 views

Likelihood values from Sigmoid

There are multiple doubts of mine associated around this theme: In MLE, we try to find the PDF parameters ($\theta$) which maximise the likelihood of the observed data ($L(\theta | data)$). To get ...
  • 344
0 votes
1 answer
84 views

Hypothesis testing and critical regions

Suppose $X$ follows a chi-squared distribution with $r$ degrees of freedom, where $r \in \mathbb{Z}^+$ is unknown. Let $X_1, X_2, ..., X_n$ be a random sample of $X$. (i) Suppose we have the following ...
user avatar
1 vote
2 answers
48 views

Is there a risk, in some special cases of likelihood form, that using logarithm in likelihood derivation, drives to losing some good solutions?

In inference statistics, one uses the quantity "likelihood" (L), and derivates it with respect to a given parameter $\theta$ (or set of parameter) in order to maximize the likelihood. Thus ...
1 vote
1 answer
87 views

Likelihood ratio test for distribution with two parameters

We let $X$ and $Y$ be independent and exponentially distributed random variable with $E(X)=\lambda_1$ and $E(Y)=\lambda_2$ where $\lambda=(\lambda_1,\lambda_2) \in {R}_+^2$ and we let $(X_1,Y_1),...,(...
  • 621
1 vote
0 answers
441 views

Fisher Information for Beta Distribution

I am trying to find the Fisher Information for $\operatorname{Beta}(\alpha,2)$. I used the following approach: $$f_X(\alpha,2)=\alpha(\alpha+1)x^{(\alpha-1)}(1-x),\alpha>0$$ $$ L(\alpha|x_i)=\...
  • 47
1 vote
1 answer
322 views

Maximum likelihood estimator doesn't exist

I was reading a paper, and the last paragraph it says For example, consider the density function $$p_{(\theta,\sigma)}(x)=\frac{1}{2\sqrt{2\pi}}e^{\large{-\frac{1}{2}(x-\theta)^2}} + \frac{1}{2\sqrt{...
  • 501
0 votes
1 answer
282 views

Maximum Likelihood Estimation with Poisson distribution

We have two independent random variables $X$ and $Y$ with $X\sim Poisson(\Phi)$ and $Y\sim Poisson(2\Phi)$, and the observations $x=2$ and $y=4$ of these. Show that the expression for the log-...
  • 377
1 vote
1 answer
34 views

Multiplying by 0 ( zero ) when finding max-likelihood estimation

I am kind of confused about the rules in math when you are solving equations I think. I am solving the following equation: $\frac{15}\lambda - \sum_{i=1}^{15} X_i = 0 $ According to maple and symbolab,...

1
2 3 4 5