Questions tagged [log-likelihood]
For questions that use the natural logarithm of a likelihood function.
237
questions
1
vote
0
answers
44
views
Deriving the log likelihood of the observed data
Hi, I am deriving the log likelihood of the observed data in part a.
How do I derive it?
Below is my solution:
$$Y_i \sim N(\mu, \sigma^2) = f(Y_i = x_i) = \phi (\xi; \mu, \sigma^2)$$ (for $r_i =1$)
$$...
- 21
0
votes
0
answers
23
views
Derivatives and Integrals of the Likelihood Function
I am reading the following notes : https://www.nan-ye.com/teach/stat3500/slides/12.pdf (page 16)
Here, it says that " The usual log-likelihood is an integral of the score function."
I have ...
- 2,360
1
vote
1
answer
32
views
Using a Bin$(k,\theta)$ random sample ($k$ known), determine the posterior distribution for $\theta$ using a Jeffrey's prior.
My question here is, am I misinterpreting a binomial random sample? Doing some research on this I'm thinking perhaps I have misunderstood something - however my answer "feels" okay ...
- 326
0
votes
1
answer
42
views
Fisher information $I(\sigma^2)$ and Cramer-Rao of estimator $S^2$
Let $X_1,...,X_n$ be a random sample with $X_1\sim N(\mu,\sigma^2)$. Compute $I(\sigma^2)$ and verify the Cramer-Rao theorem for the estimator $S^2$
Since $f_{\mu, \sigma^2} = \frac{1}{\sqrt{2\pi} \...
- 1,574
0
votes
0
answers
6
views
Variational Bayes on Gaussian process Kernel hyper-parameter estimation
I have a Gaussian process prior model for a signal $y = \mathcal{GP}(0, K)$. The kernel has two hyper-parameters $\mu$ and $\sigma$.
$$ K(t_1, t_2) = g(\sigma, t_1-t_2) h(\mu, t_1-t_2) = \exp\left(-\...
- 113
2
votes
1
answer
70
views
Multinomial Logistic Regression likelihood
Suppose we have the following parametric model for logistic regression:
$$\phi_{i} = \frac{\exp{(a^{T}x_{[i]}})}{\sum_{k = 1} ^{M} \exp{(a^{T}x_{[k]})}}$$for $i = 1, \dots, M$ and that the parameter ...
- 165
0
votes
1
answer
38
views
Partial derivative of a Likelihood function
I'm trying to get the partial derivatives $\frac{\partial L}{\partial w}$ of a log-Likelihood function
$$
L(w) = \sum_{n=1}^{N}\sum_{k=1}^{K}y_{nk}\cdot log(\frac{e^{\sum_{i=1}^{D}w_{ki}x_{i}}}{\sum_{...
- 1
0
votes
0
answers
91
views
EM algorithm for exponential distribution
Suppose $W$ is a nonnegative random variable having an exponential distribution with mean $\mu$.$\newcommand{\exp}{\operatorname{exp}}$
\begin{align}
f(w ; \mu) = \frac{1}{\mu} \exp(-w/\mu)I_{(0,\...
- 277
3
votes
0
answers
15
views
Family of transformations can I find a density function?
Let's consider the family of transformations given by
$$g_a(Y)=\begin{cases}
\frac{e^{aY}-1}{a} & \text{ for } a\neq 0 \\
Y & \text{ for } a=0
\end{cases}$$
for $Y\in\mathbb{R}$. Analogous to ...
- 490
1
vote
1
answer
87
views
MLE for a Borel distribution
I would appreciate help with on how to find the estimation $\hat{\beta}_{ML}$ for a Borel distribution. I am doing something wrong in, my guess, the likelihood function and therefore the final answer ...
- 379
0
votes
0
answers
19
views
how to simplify a log likelihood function
How to simplify/ get the log likehood of this?
I understand basically the idea is to get the LN of both sides and simplify??
- 101
0
votes
0
answers
36
views
negative sign of the Log-likelihood gradient
I was watching an explanation about how to derivate the negative log-likelihood using gradient descent, Gradient Descent - THE MATH YOU SHOULD KNOW but at 8:27 says that as this is a loss function we ...
1
vote
0
answers
24
views
Show that a constrained probit likelihood is globally concave
I have a constrained probit likelihood and I would like to know if it is globally concave.
My problem can be simplified as follows. Let $n\in\mathbb{N}^{\ast}$. Let $F$ be a function defined on $\...
- 371
0
votes
0
answers
55
views
Calculate maximum likelihood of Ridge regression
I'm reading a bit about ridge regression,
$Ridge= \sum_{i=1}^{n}\left(y_i-\beta_0-\sum_{j=1}^{p}\beta_jx_{ij}\right)^2+\lambda\sum_{j=1}^{p}\beta_j^2$
and I wanted to try and implement it. So I ...
- 153
0
votes
0
answers
16
views
Where does discrete probabilities in Forward-Backward algorithm for Hidden Markov Models come from?
I am trying to derive a Forward-Backward algorithm used in Hidden Markov Models to compute the likelihood $P(x | \theta)$ that sample $x = (x_1, ... x_n)$ comes from HMM defined by set of parameters $\...
- 241
3
votes
1
answer
59
views
Quadratic Approximation for Log-Likelihood Ratio Processes
I'm trying to understand why the quadratic equation can approximate the log likelihood ratio, and how it is derived: $$\mathrm{Log}(\mathrm{LR})=\frac{1}{2}\left(\frac{\mathrm{MLE}-\theta}{S}\right)^2$...
- 33
0
votes
0
answers
52
views
Mixture of Markov Models question
This is a follow up from this question.
Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in ...
- 29
1
vote
0
answers
23
views
Chaining a conditional likelihood function
Before I proceed, allow me to apologise in advance for potential abuse of notations. Let $\{x_t:\Omega \to \mathbb{R}\}_{t=1,\cdots,n}$ be a stochastic process, defined on a probability space $(\Omega,...
- 173
0
votes
1
answer
63
views
is the mix of convex and linear functions always convex function?
I want to prove that the following composed function $g \circ L$ is always (strictly) convex :
\begin{alignat*}{3}
&g&&(t&&) && =-\log(1-e^{-t}) \qquad && \text{...
- 103
0
votes
0
answers
21
views
Negative log-likelihood of Gaussian distributions
In a paper about collective outlier detection, I found the following penalized cost formula.
$$ \sum \limits_{t\notin \cup \left[{\tilde{s}}_i+1,{\tilde{e}}_i\right]}\mathcal{C}\left({\mathbf{x}}_t,{\...
- 2,646
0
votes
0
answers
20
views
Does the negative log likelihood of Discrete Weibull have Lipschitz gradients?
I am working on different convergence analyses of optimization algorithms for machine learning. However, almost all of them are based on the assumption that the objective function $f(x)$ has Lipschitz ...
0
votes
0
answers
21
views
Beta binomial regression model: sufficient statistic of the regression coefficients
I have the following beta-binomial with logit link function model:
$$f(y_i|\pi_i,\phi)\sim\operatorname{binomial}(p_i), \text{with }p_i \sim \operatorname{beta}(\frac{\pi_i
}{\phi},\frac{(1-\pi_i)}{\...
0
votes
0
answers
18
views
Is there a probabilistic concept or theory for infinitesimal logarithmic product interpretation of integral?
If we have a number of independent events in probability, we can calculate it's likelihood :
$$\prod_{\forall i} p_{i}$$
We can also consider ( where $H$ is the Heaviside step function )
$$\int L(t) H(...
- 25k
0
votes
0
answers
18
views
MLE & MLP Comparison
I am studying the allele frequency in population genetics and I have some questions:
When using a conjugate prior and comparing the MLE and MLP results for allele frequency, how does the mean squared ...
- 1
1
vote
0
answers
27
views
Differentiation of logLikelihood MLE
Given a log likelihood function,
$$l=\sum_{i=1}^m\log[\frac{n!}{X_{i,1}!X_{i,2}!(n-X_{i,1}-X_{i,2})!}p_1^{X_{i,1}}p_2^{X_{i,2}}(1-p_1-p_2)^{n-X_{i,1}-X_{i,2}}]$$
To derive: with respect to $p_1$, $\...
- 21
0
votes
0
answers
50
views
Cross entropy and log-likelihood
I think it's pretty clear to me that average log-likelihood is equivalent to negative cross-entropy for discrete distributions, as shown here:
$$\frac{1}{N}\log\mathcal{L}(\theta) = \frac{1}{N}\log \...
0
votes
1
answer
89
views
How to find the value that give the global maximum of a function?
I have got the following function:
$$L(x,y) = \frac{(16.2x +0.9y+5.2)^{24} e^{-(16.2x +0.9y+5.2)}}{24!} \cdot \frac{(2.1x +4.2y+0.9)^8 e^{-(2.1x +4.2y+0.9)}}{8!} \tag{1}$$
and I am attempting at ...
- 45
0
votes
0
answers
39
views
How to find the global maximum of a likelihood-function?
I have an expression for a likelihood function,$L_{A,B}$, with two parameters, $(A,B)$.
I want to find the global maximum for this function, how can this be done?
A colleague has told me that:
"...
- 143
0
votes
1
answer
29
views
Simplifying $-\log\left(\left( \frac{1}{2\pi\sigma^2}\right)^{n/2} \exp \left( -\frac{1}{2\sigma^2} \sum_{i=1}^{n}(\mu-y_i)^2\right)\right)$
I'm not the most mathematically minded, but I'm doing my best to learn MLEs. We were provided the following original likelihood function of a Normal Distribution and I am having trouble understanding ...
0
votes
1
answer
345
views
Derivate of the the negative log likelihood with composition
We want to solve the classification task, i.e., learn the parameters $\theta = (\mathbf{W}, \mathbf{b}) \in \mathbb{R}^{P\times K}\times \mathbb{R}^{K}$ of the function $f_\theta: \mathbb{R}^P \to [0, ...
- 201
0
votes
0
answers
42
views
Likelihood-function : How to simplify it?
The fracture strength of hard bricks satisfies an Erlang distribution of order $2$. There are $n \in \mathbb{N}$ fracture strength tests that are carried out.
We consider the statistical product model ...
- 13.5k
1
vote
1
answer
110
views
Show that expression is Gamma distribution with given parameters
I had the expression $$\frac{1}{p(y)}(n\lambda)^k e^{-n \lambda}$$
for $k \in \{0,1,2,...\}$ where we here have that $k=\sum_{i=1}^n y_i$ where I think I can ignore $p(y)$, but if not it is given by $...
- 645
1
vote
1
answer
191
views
Negative Log likelihood and Derivative of Gaussian Naive Bayes
I am trying to derive negative log likelihood of Gaussian Naive Bayes classifier and the derivatives of the parameters.
So there are class labels $y \in {1, ..., k}$, and real valued vector of $d$ ...
- 57
1
vote
3
answers
58
views
All solutions to $f(a,b)+f(b,a)=0$
I am looking for all functions $f:(0,1)^2\to\mathbb{R}$ that satisfy $f(a,b)+f(b,a)=0$ for every $(a,b)\in(0,1)^2$. I know that $f\equiv 0$, $f(a,b)=c(a-b)$, and $f(a,b)=\pm\log(a/b)$ are such ...
- 55
1
vote
1
answer
87
views
Likelihood of p and q given Y
Suppose that Jon has a coin with a probability of landing on heads p and Ann has a coin with a probability of q landing on heads.
Each time they flip a coin and they write the sum of the outcome ...
- 11
1
vote
2
answers
335
views
MLE + Cramér-Rao bound of a discrete random variable
Can someone explain how to compute the maximum likelihood estimator, the Fisher information and the Cramér-Rao bound of a discrete random variable please?
I came across this exercise while reviewing ...
- 15
0
votes
2
answers
79
views
Roots of log binomial likelihood equation
I have been wondering if there is a way to approximate the roots of the log binomial likelihood equation. To be clear the equation is
$$a \cdot \log\left(x\right) + b \cdot \log\left(1 - x\right) = t$$...
- 3
0
votes
0
answers
18
views
Formulate a likelihood function for an uncertain discrete "n", where y is binom(n,p), and we know y and p
The problem goes as followed: multiple balls have been tossed into a box, but we do not know how many that were tossed. However, we know that 5 of the balls that were tossed had a blue colour ($y=5$), ...
3
votes
1
answer
299
views
When does sup and function commute?
For $f,g$ real-valued functions, $f$ weakly increasing and continuous, $A\subseteq \mathbb{R}$, can we say
\begin{align*}
\sup_{x\in A}f(g(x))=f(\sup_{x\in A}g(x))
\end{align*}
I ask because I notice ...
- 11.1k
1
vote
0
answers
32
views
Log likelihood from multi-variate Normal distribution?
If we consider a linear mixed model that consider fixed effect and random effect: $$y = X\beta + W_uu + e,$$
where
$n$ is the sample size
$y \in \mathbb{R}^{n \times 1}$ is the response vector
$X \in ...
- 751
1
vote
0
answers
88
views
Differentiating log likelihood with respect to data
I'm working with the probabilistic model derived from the KL NMF, in which we sample each entry of a data matrix X from the distribution
$\forall_{i,j},X_{ij}\sim\text{Poisson}\left(\left[WH\right]_{...
- 11
0
votes
0
answers
44
views
Log-Likelihood of Piecewise-Defined Function
This is a follow-up to this question, which might be too difficult. So I break the problem down. What I need to find is the log-likelihood of the following function:
$$f(x, a_{sg}, a_{pl}, p) = \begin{...
- 13
1
vote
1
answer
100
views
Log-Likelihood Function of Piecewise-Defined Function
I'm a data analyst, no mathematician. Mostly I do 'standard' stuff such as linear mixed-effects regressions or generalized additive mixed models. But now I need to determine the maximum log-likelihood ...
- 13
1
vote
1
answer
127
views
Likelihood ratio hypothesis test for the exponential distribution.
I am trying to understand the following logic that I found on the internet that implements a hypothesis test on the exponential distribution using the likelihood ratio test.
We want to test the ...
1
vote
1
answer
41
views
How to find the log-likelihood for this density?
We let $(X,Y)$ be stochastic variables with values in $N × R$ determined by $ X = x $ being subtracted from one
Poisson distribution with mean $\lambda$
after which Y is subtracted from a gamma ...
- 645
1
vote
1
answer
149
views
Likelihood values from Sigmoid
There are multiple doubts of mine associated around this theme:
In MLE, we try to find the PDF parameters ($\theta$) which maximise the likelihood of the observed data ($L(\theta | data)$). To get ...
- 354
0
votes
1
answer
136
views
Hypothesis testing and critical regions
Suppose $X$ follows a chi-squared distribution with $r$ degrees of freedom, where $r \in \mathbb{Z}^+$ is unknown. Let $X_1, X_2, ..., X_n$ be a random sample of $X$.
(i) Suppose we have the following ...
user902292
1
vote
2
answers
49
views
Is there a risk, in some special cases of likelihood form, that using logarithm in likelihood derivation, drives to losing some good solutions?
In inference statistics, one uses the quantity "likelihood" (L), and derivates it with respect to a given parameter $\theta$ (or set of parameter) in order to maximize the likelihood. Thus ...
1
vote
1
answer
110
views
Likelihood ratio test for distribution with two parameters
We let $X$ and $Y$ be independent and exponentially distributed random variable with $E(X)=\lambda_1$ and $E(Y)=\lambda_2$ where $\lambda=(\lambda_1,\lambda_2) \in {R}_+^2$ and we let $(X_1,Y_1),...,(...
- 645
1
vote
0
answers
857
views
Fisher Information for Beta Distribution
I am trying to find the Fisher Information for $\operatorname{Beta}(\alpha,2)$. I used the following approach:
$$f_X(\alpha,2)=\alpha(\alpha+1)x^{(\alpha-1)}(1-x),\alpha>0$$
$$ L(\alpha|x_i)=\...
- 47