I just learned about likelihood from Wikipedia. It says that the likelihood of outcome $X$ given parameters $θ, L(X|θ)$ -- equals the probability of parameters θ given $X, P(θ|X)$.

I am trying to understand this paper, which on page 2 seems to talk about the likelihood of a sample (p-tilde) given the model $p$

enter image description here

To me, the statement above seems to translate to "the likelihood of an observed sample (p-tilde) given the model p is equivalent to the sum of all empirically observed likelihoods of each symbol $y *$ the $log$ of the probability of $y$.

But how does that have to do with the wikipedia definition of likelyhood? Does the model, p, implicitly include the parameters of the model, θ? Also, do the three bars in the equation above indicate that the thing on the left is monotonically related to the thing on the right? Wikipedia on monotonic functions does not mention this notation.


The equation (3) is defining "the average per-symbol log-likelihood." That's what the symbol $\equiv$ means here.

So what the paper is saying is that $\text{Pr}(\tilde{p}|p)$ (which is "the probability (or likelihood) of the sample $\tilde{p}$") is monotonically related to $L(\tilde{p}|p)$ (which is "the average per-symbol log-likelihood"). Your confusion seems to stem from the fact that the author has used the letter $L$ here. But $L$ does not always stand for likelihood. He is telling you that the function $L$ stands for "the average per-symbol log-likelihood," and then claiming that the function $\text{Pr}$ (the likelihood) is monotonically related to the function $L$.

The whole point of this is that, because of monotonicity, maximizing the function $\text{Pr}(\tilde{p}|p)$ is equivalent to maximizing the function $L(\tilde{p}|p)$, but $L$ is easier to work with.

There is no discrepancy with Wikipedia's definition of likelihood. Note the italicized phrase in parentheses after the word "probability."

  • $\begingroup$ "Note the italicized phrase in parentheses after the word "probability." do you mean in the paper or the wikipedia? Can you point to the phrase you are talking about? $\endgroup$ – bernie2436 May 5 '14 at 13:00
  • $\begingroup$ I quoted the phrase! It's from the paper! Read carefully! $\endgroup$ – symplectomorphic May 5 '14 at 13:01
  • $\begingroup$ it is from the sentence that introduces equation (1). $\endgroup$ – symplectomorphic May 5 '14 at 13:02
  • $\begingroup$ yeah I see it now. Thanks $\endgroup$ – bernie2436 May 5 '14 at 13:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.