Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [log-likelihood]

For questions that use the natural logarithm of a likelihood function.

1
vote
0answers
14 views

The profile likelihood for a Weibull distribution

I have an iid sample $Y_1, Y_2, ..., Y_n$ of a Weibull$(\alpha, \beta)$ with the density given by $f(y)=\beta \alpha y^{\alpha-1} exp(-\beta y^\alpha)$, with $\alpha, \beta>0$ and $\theta=(\alpha,\...
2
votes
1answer
59 views

Example of “The eigenvalues of data covariance matrix, $\Phi^T\Phi$ measure the curvature of the likelihood function.”

I am reading PRML, Chapter 3.5.3, screen shot attached. I can understand the derivation and maths but hard to understand the meaning of "The eigenvalues of data co-variance, $\Phi^T\Phi$ matrix ...
0
votes
0answers
14 views

Log-likelihood of gamma function is concave!

I know that one of conditions for uniquely existence of MLE is strict log-concavity of the log likelihood. So, if $X_i$'s are i.i.d samples from $Gamma(\alpha,\beta), i=1,2,...,n$, the log-likelihood ...
0
votes
0answers
38 views

Finding Fisher information

Let $X$ distribution belongs for the family $\mathcal{P}\{P_{\theta}, \theta \in \Theta \}$. We need to find Fisher information $I(\theta)$ according $n$ simple sample, when $P_{\theta}$ is $N(\mu,\...
0
votes
1answer
34 views

Confusion with maximizing likelihood of binomial parameter $p$

Consider the following data from a sample of a binomial distribution $X$ with $n=2$ and unknown parameter $p$: $$P(X=0)=.175,\ \ P(X=1)=.45,\ \ \text{and} \ \ P(X=2)=.375.$$ My goal is to find the ...
1
vote
1answer
70 views

How is the log-likelihood for a multinomial logistic regression calculated?

In a multinomial logistic regression, the predicted probability $\pi$ of each outcome $j$ (in a total of $J$ possible outcomes) is given by: $ \pi_j = \frac{e^{A_j}}{1+\sum_{g \neq j}^Je^{A_j}} $ ...
0
votes
0answers
126 views

MLE of Parameters of Bivariate Normal Distribution

I am working through find the maximum likelihood estimators of the bivariate normal distribution, without using matrices. I have the following density function: $f(Y_1,Y_2) = \frac{1}{2\pi\sigma_1\...
0
votes
1answer
33 views

method of moments and maximum likelihood estimators

I'm looking to find the estimate of $\mu$ for $n$ data using the method of moments and the maximum likelihood for the pdf given by $f(x) = \begin{cases} e^{-(x-\mu)}, & \text{if} \, x \geq \mu \\ ...
1
vote
0answers
16 views

Weighted maximum likelihood estimator for poisson variables

I have been learning about the E-M algorithm in the Normal case. I would like to create the algorithm with two Poisson variables. To do so, I need to calculate the weighted MLE as seen here in the M-...
0
votes
0answers
26 views

how log likelihood's derivative is equal to zero in maximum log likelihood.

if the log-likelihood function is strictly increasing and it has not horizontal asymptote then how it's derivative is equal to zero in maximum log likelihood. Now since it is strictly increasing every ...
0
votes
1answer
55 views

Plot the log likelihood function for X ~ Bin(10, p)

This is the third part of a question I am working on. So after solving for the parameter estimate $p$ by using the MLE method, I've been asked to plot the log likelihood function for X = 5, n = 10. To ...
0
votes
0answers
13 views

Find the MLE of a parameter with a given density function [duplicate]

I have to find the MLE of $\theta$ The density function is given, which is: $$f(x) = \theta x^{-2}$$ And $x \geq \theta, 0 $ for $ x < \theta, \theta > 0 $ unknown I have tried to find the ...
0
votes
2answers
23 views

Does the Integral over the product of distributions result in another distribution?

In this post one guy asks why those likelihoods in Maximum Likelihood get multiplied. It's clear why they get multiplied. I have another problem. In my textbook it is state that for each sample the ...
0
votes
0answers
21 views

Log likelihood of normal time series

Given a time series $\{x_i, i=1, \ldots, n\}$ and define the corresponding series $$y_i = \log(x_{i+1} + a) - \log(x_i + a),$$ where we assume that $y_i \sim N(\mu, \sigma^2)$. The aim is to estimate $...
0
votes
0answers
14 views

bounding min-entropy gain in differential privacy

In privacy-related computer science literature, we say that a randomized algorithm $\mathcal{K}$ that produces a model $\theta$ from a sample $X=(x_1,...,x_n)$ is $\epsilon$-differentially private iff ...
0
votes
1answer
123 views

find partial derivative of the function

I have this equation which is basically a maximum likelihood equation for EM-algorithm. $$L(\theta) = \sum_{i=1}^n{\ln{(\sum_{j=1}^kw_jp_j(x_i;\theta_j))}}$$ I'm trying to derive a partial derivative ...
0
votes
0answers
52 views

Quadratic expansion of log-likelihood

From the paper I'm reading: Let $\mathbf{x}$ represent the distribution of linear attenuation coefficient of an object and $[\mathbf{Hx}]_m$ represents their line integral. The $m$th CT measurement, $...
0
votes
0answers
19 views

Log-likelihood function reference

I am learning a code that uses CUSUM to detect changes. The code uses log-likelihood function: logp = stepsize*basesd/variance * (data[k] - mean - stepsize*basesd/2.) (instantaneous log-likelihood ...
1
vote
0answers
106 views

Convexity of a Log Likelihood Function

Goal I would like to proof than the Negative Log Likelihood Function of Sample drawn from a Normal Distribution is convex. Below a Figure showing an example of such function: Motivation of this ...
0
votes
1answer
52 views

Taylor expansion of likelihood function

$\require{\cancel}$ ...For large samples, as a consequence of the central limit theorem, the likelihood function approaches a gaussian, whose expected value is equal to the maximum likelihood ...
0
votes
0answers
58 views

2D Maximum Likelihood Fit

I have read a couple of places that it is possible to do a 2D (or 3D) maximum likelihood fit, but I can't seem to understand how this would work. Suppose I'm considering a probability distribution ...
1
vote
0answers
93 views

Log-likelihood of zero-truncated Poisson

Question: Edwards and Eberhardt (1967) conducted a live-trapping study on a confined population of known size. In their study, wild cottontail rabbits were penned in a 4-acre rabbit-proof enclosure. ...
0
votes
0answers
22 views

Monte Carlo Markov Chain - Metropolis-Hastings - Estimation of parameters

I have 6 parameters to estimate : $p=(\theta=[a,b]$, $\nu=[r_0,c_0,\alpha,\beta])$ with Bayesian and MCMC methods : $$\text{PSF}(r,c) = \bigg(1 + \dfrac{r^2 + c^2}{\alpha^2}\bigg)^{-\beta}$$ and the ...
0
votes
1answer
5 views

How to introduce different costs by class in a binary logistic regression?

What is the form of the Negative Log-Likelihood Goal function in Logistic regression if we introduce different costs per class (e.g. the cost of erring class 1 is errc1, and class 2 is errc2)?
0
votes
0answers
16 views

Expectation Maximization (EM) : find all parameters from a PSF (Point Spread Function)

I have the 2 parameters arrays : $\theta=[a,b]$, $\nu=[r_0,c_0,\alpha,\beta]$ with the distribution (a point spread function = PSF = response of a Dirac) : $\text{PSF}(r,c) = \bigg(1 + \dfrac{r^2 + c^...
0
votes
0answers
39 views

Log-likelihood function for lambda

If x1, x2, . . . , x6 are i.i.d. Poisson observations with mean λ. Furthermore,y1, y2, . . . , y8 are i.i.d. N(ln(λ), 1) observations, independent of the xi’s. What will be the log-likelihood ...
0
votes
0answers
24 views

Estimation of 2 parameters with Maximum likelihood and a function depending on 2 random variables

I have the following PSF (Point Spread Function) (Moffat PSF) : I want to estimate the parameters $\alpha$ and $\beta$ ($\theta=[\alpha,\beta]$ represents the vector of parameters to estimate) with ...
0
votes
0answers
11 views

log-likelihood applied to classification of image histograms

I have a reference histogram of an 8-bit image, and histograms of multiple test cases for which i'm trying to determine if they belong to the same reference class. If I understand the procedure, for ...
0
votes
1answer
145 views

Poisson regression log likelihood function given sample data

If $Y_i$ has a $\textrm{Poisson}(\lambda_i)$ distribution then it has density function, $$p(Y_i|\lambda_i) = \frac{\lambda_i^{Y_i}}{Y_i!}e^{−\lambda_i}$$ Suppose we think the variables $Y_1, \dots , ...
0
votes
0answers
22 views

Finding maximum likelihood estimator.

Can someone please let me know if my solution is okay? I feel as though I got it wrong, so here is my work: $L(\theta, \mathbf{x})=\prod _{i=1}^{n}(\theta + 1)x_{i}^{-\theta -2}$ $=(\theta + 1)^{n} ...
0
votes
1answer
315 views

Analytical form of Jeffrey's prior

Derive, analytically, the form of Jeffery's prior for $p_J(\lambda)$ for the parameter $\lambda$ of a Poisson likelihood, where the observed data $y = (y_1, y_2,...,y_n)$ is a vector of i.i.d draws ...
0
votes
0answers
48 views

Weird visual result for MLE of normal distribution

I am trying to visualize the log likelihood estimator for normal distribution via python but in vain. I am not sure if its a python issue or formuala issue in the code. Can you kindly have a look? ...
0
votes
1answer
282 views

Hessian of negative log-likelihood of logistic regression is positive definite?

I'm trying to show that the Hessian of the negative of the log likelihood with two parameters is positive definite, but I'm not sure how to go about it once I compute the Hessian. The function is: $-...
0
votes
0answers
86 views

What is the posterior distribution of a Pareto distribution?

I want to derive the posterior of the Pareto distribution, but I can't seem to make it work. I know that Pareto is given by: $p(\theta |\alpha,\beta) = \dfrac{\alpha\beta^\alpha}{\theta^{\alpha+1}} $ ...
2
votes
1answer
36 views

probability of false alarm for a loaded die

A loaded 6-sided die has probability 1/4 for 3 & 4 and 1/8 for 1,2,5,6. If i decide whether a die is loaded or not based on one roll what is the probability of falsely classifying a fair die as ...
1
vote
0answers
73 views

Low-rank approximation of covariance matrix

I am reading a paper in which the author expresses the log-likelihood function for a gaussian as $L(\Theta^{(k)}) = -N$ log det $R^{(k)}$ - tr $\left[{R^{(k)}}^{-1} \hat{R}\right]$ where $N$ is ...
1
vote
0answers
39 views

Gradient of the loglikelihood for the RSM (contrastive divergence)

I'm actually implementing an RSM in TensorFlow and I've realized that when I used the energy function and let TensorFlow compute the gradient, the ouput of my RSM (Replicated Softmax Model) is pure ...
0
votes
0answers
35 views

Detection of Gaussian Signal in Gaussian Noise

I am constructing a hypothesis test for detection of zero-mean Gaussian signals in zero-mean Gaussian noise under $L$ number of observations, where the noise variance is known but the signal variance ...
0
votes
0answers
30 views

Calibrating the likelihood ratio test

Background Let $x_1,....x_n$ be real-valued random variables that are distributed according to either measure $P_0$ or $P_1$, and let these measures have densities $f_0$ and $f_1$ respectively. The ...
1
vote
0answers
29 views

Understanding terminology of Mixtures of Gaussians model

I'm following this lecture notes to understand mixture of Gaussian model and EM (Expectation Maximization) algorithm to fit it. I understand the complete intuition behind this algorithm, which is ...
0
votes
0answers
24 views

Calculating the variance of a normal distribution given the log-likelihood of each point

A random variable $X$ is normally distributed: $X \sim \mathcal{N}(\mu, \sigma^2)$ Thus, the probability density function is: $f(x \mid \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \mathrm{e} ^{-...
0
votes
1answer
45 views

Confusing summation of $i$ not $j$, but $j$ is not defined

I am trying to write out the following log-likelihood: $$\mathcal L(\vec{x}, \vec{y}) = \sum_{i} \left[ k_{i}^{out} (\boldsymbol{A}^*) \ln x_i + k_i^{in} (\boldsymbol{A}^*) \ln y_i\right] - \sum_{i \...
0
votes
0answers
12 views

Likelihood function of linear model with normal distribution

Suppose the following observation, $$ X[n] = Ar^n + W [n]~~~~~~~~~~~~~~~~~~~~~n=0,1,\cdots, N-1$$ W[n] is i.i.d normal distribution as $N(0, \sigma^2)$. $r$ is constant and $A$ is the parameter to ...
1
vote
1answer
138 views

Maximum Likelihood Estimator of : $f(x) = \theta x^{-2}, \; \; 0< \theta \leq x < \infty$

Exercise : Find a maximum likelihood estimator of $\theta$ for : $f(x) = \theta x^{-2}, \; \; 0< \theta \leq x < \infty$. Attempt : $$L(x;\theta) = \prod_{i=1}^n \theta x^{-2} \mathbb{I}_{...
2
votes
2answers
61 views

log likelihood function and MLE for binomial sample

Let $X_1,X_2,...;X_n$ be a random sample with $X_i$~$Binomial(m,p)$ for $i=1,...,n$ and $m=1,2,3,...$ and let $p\in (0,1)$. We assume $m$ is known and we are given the following data $x_1,...,x_n\in\{...
1
vote
1answer
74 views

Large sample confidence intervals

The Question: Let $x_1,\dots,x_n$ be a large sample taken from a distribution with density function $f(x;\theta)$ where $\theta$ is a scalar parameter. Let $\ell (\theta)$ be the log-likelihood of ...
0
votes
0answers
52 views

Find the Maximum Likelihood Estimator for covariance matrix in mixed model

I have a random effects-only linear model as follows: $$\mathbf{y_i = Z_ib_i + \epsilon_i,\quad i=1,...,N}$$ $$\mathbf{\epsilon_i \sim N(0,\sigma^2I)},\quad \mathbf{b_i \sim N(0,\sigma^2D)}$$ where $...
1
vote
0answers
24 views

likelihood function of multivariate logit normal incorrect?

Wikipedia gives the equation for the likelihood function of the multivariate logit normal distribution as follows: In the case of $\mathbf{x}$ with sigmoidal elements, that is, when: $$\mathbf{y} ...
0
votes
1answer
33 views

Simplify the log of the multivariate logit (or logistic)-normal probability density function

According to wikipedia, the probability density function of the multivariate logit-normal (sometimes called logistic-normal) distribution is $$ f_X( \mathbf{x}; \boldsymbol{\mu} , \boldsymbol{\Sigma} ...
1
vote
2answers
76 views

Find MLE estimators of PDF

I am quite stumped by the following problem. The usual log-likelihood route with differentiation doesn't work. The problem is as follows: Let $X_1,...,X_n$ be i.i.d. random variables from a ...