Let $\mathbf{X} = (X_1, \ldots, X_n)$ be a sample of i.i.d. random variables ($X_i \sim P_\theta$) and $\mathbf{x}=(x_1,\ldots,x_n) \in \mathbb{R}^n$ – its realization.
I understand the following definition of the likelihood function:

$$L(\mathbf{x}; \theta) = \prod_{i=1}^n f(x_i; \theta).$$

Here likelihood function is just a value of a joint probability density (joint pmf in discrete case) of a random vector $\mathbf{X}$ (where $X_i \sim P_\theta$) calculated at point $\mathbf{x}$.

But in some statistics textbooks the definition of the likelihood function is given as follows:

$$L(\mathbf{X}; \theta) = \prod_{i=1}^n f(X_i; \theta)$$

i.e. in this case likelihood function is a random variable because $X_i = X_i(\omega)$ are random variables.

Ok, I can interpret $L(\mathbf{X}; \theta)$ as a random variable in the case of an absolutely continuous random vector $\mathbf{X}$.

But what is the correct interpretation of $L(\mathbf{X}; \theta)$ in the case of discrete $\mathbf{X}$?
I think in this case we have ($p$ is a pmf) $$L(\mathbf{X}; \theta) = \prod_{i=1}^n p(X_i; \theta) = \prod_{i=1}^n P_\theta(X_i = X_i) = 1.$$

But this expression is meaningless because it is always equal to one...

  • $\begingroup$ Both those definitions are same. Likelihood function is taken as the joint pdf/pmf given the sample $\mathbf X$ (or $\mathbf x$). It is a function of $\theta$, not a random variable. $\endgroup$ – StubbornAtom May 18 '18 at 12:26
  • $\begingroup$ @StubbornAtom I disagree. We write $L(\mathbf{x}, \theta)$ when we seek an ML-estimate and we write $L(\mathbf{X}, \theta)$ when we seek an ML-estimator. That things are not the same. Also look at the definition for the Fisher information. In the last definition you can't simply replace $X$ by $x$ in the argument of log-likelihood function. $\endgroup$ – Rodvi May 18 '18 at 12:37
  • 1
    $\begingroup$ Using the first definition, if I find the ML estimate of $\theta$, say $\hat\theta=T(\mathbf x)$ , then the ML estimator would be simply $T(\mathbf X)$. That is , if you want to distinguish between estimate and estimator by using small and capital letters respectively. $\endgroup$ – StubbornAtom May 18 '18 at 12:45
  • 2
    $\begingroup$ Your error is in the next to last equality. $p(X_{i};\theta)$ is a random variable, not a scalar and in particular not the constant one. $\endgroup$ – Brian Borchers May 18 '18 at 14:18
  • $\begingroup$ @BrianBorchers Yes, you are right. In other words, $p(X_i; \theta)$ is not a pmf value anymore (not a value of probability measure $P_\theta$), it is a random variable. It has the same "visual form" as pmf $p(x_i; \theta)$ but its properties are different. Similarly, $f(X_i; \theta)$ is not a pdf value, it is a random variable. Thank you! $\endgroup$ – Rodvi May 18 '18 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.