I am reading the following notes : https://www.nan-ye.com/teach/stat3500/slides/12.pdf (page 16)
Here, it says that " The usual log-likelihood is an integral of the score function."
I have never heard of this relationship before and am trying to understand where it comes from
To provide some context - assume we have a set of independent and identically distributed (i.i.d) random variables $X_1, X_2, ..., X_n$, with a common probability distribution $f(x|\theta)$, where $\theta$ is the unknown parameter we want to estimate.
The likelihood function is the joint probability density function (PDF) of the data, given the parameter $\theta$, i.e.,
$$ L(\theta \mid x_1, x_2, ..., x_n) = f(x_1, x_2, ..., x_n \mid \theta) = \prod_{i=1}^n f(x_i \mid \theta) $$
The log-likelihood function is simply the natural logarithm of the likelihood function:
$$ \ell(\theta \mid x_1, x_2, ..., x_n) = \log L(\theta \mid x_1, x_2, ..., x_n) = \sum_{i=1}^n \log f(x_i \mid \theta) $$
The score function, also known as the gradient of the log-likelihood function, is given by:
$$ \frac{\partial \ell(\theta \mid x_1, x_2, ..., x_n)}{\partial \theta} = \sum_{i=1}^n \frac{\partial \log f(x_i \mid \theta)}{\partial \theta} $$
Thus, using the basic principles of Calculus (i.e. integral-derivative relationship) - is this why "The usual log-likelihood is an integral of the score function"?
Thanks!
