Last week during statistics class, my professor asked us a few basic questions about statistics. We could answer most of them except these three questions that we could not provide him good answers. Here are the questions:

  1. Why must we maximize the likelihood function? Why we do not minimize it like sum of square error? What is the fundamental reason behind maximizing likelihood function?
  2. What is the philosophy of likelihood function formula (below). Why must we multiply the pdf's? Why do we not add them? $$ \begin{align} L(\Theta|x_i)=\prod_{i=1}^n f(x_i|\Theta) \end{align} $$
  3. If we throw a coin ($C$) or maybe we roll a die ($D$), what is its population? Is it $C=\text{{Head, Tail}}$ for a coin or $D=\text{{1, 2, 3, 4, 5, 6}}$ for a die?

He only gave us three keywords: sample, parameters, and population to answer first two questions and we do not know how these keywords could help us answer those questions. We could just not believe ourself because almost everyday we deal with statistics problems but could not answer these 'simple' questions. Could anyone here give me good answers or perhaps the best explanations to answer those questions? I would be grateful for any help you are able to provide.


1) The data $x_i$ is assumed to have been generated by the distribution $f_\Theta()$. The maximization of the likelihood gives the $\Theta$ which is most likely to have generated the observed data.

2) The $x_i$ are independent so the basic laws of probability are just being applied when the pdf are multiplied. Just like the probability of two heads is $\frac{1}{2} \times \frac{1}{2}$. Each term $f(x_i|\Theta)$ is the probability of the data generating process delivering a $x_i$. (True for discrete distribution, same idea for continuous x, but need to be a bit more careful.)

3) Correct

  • $\begingroup$ For no. 2 and 3, we did answer like that but it seemed our professor was not satisfied yet. For no. 1, let me take it first as a consideration. Thanks for your answer. I really appreciate it. $\endgroup$
    – Tunk-Fey
    Mar 19 '14 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.