I am reading the following notes : https://www.nan-ye.com/teach/stat3500/slides/12.pdf (page 16)

Here, it says that " The usual log-likelihood is an integral of the score function."

I have never heard of this relationship before and am trying to understand where it comes from

To provide some context - assume we have a set of independent and identically distributed (i.i.d) random variables $X_1, X_2, ..., X_n$, with a common probability distribution $f(x|\theta)$, where $\theta$ is the unknown parameter we want to estimate.

The likelihood function is the joint probability density function (PDF) of the data, given the parameter $\theta$, i.e.,

$$ L(\theta \mid x_1, x_2, ..., x_n) = f(x_1, x_2, ..., x_n \mid \theta) = \prod_{i=1}^n f(x_i \mid \theta) $$

The log-likelihood function is simply the natural logarithm of the likelihood function:

$$ \ell(\theta \mid x_1, x_2, ..., x_n) = \log L(\theta \mid x_1, x_2, ..., x_n) = \sum_{i=1}^n \log f(x_i \mid \theta) $$

The score function, also known as the gradient of the log-likelihood function, is given by:

$$ \frac{\partial \ell(\theta \mid x_1, x_2, ..., x_n)}{\partial \theta} = \sum_{i=1}^n \frac{\partial \log f(x_i \mid \theta)}{\partial \theta} $$

Thus, using the basic principles of Calculus (i.e. integral-derivative relationship) - is this why "The usual log-likelihood is an integral of the score function"?




You must log in to answer this question.

Browse other questions tagged .