Maximum Likelihood Estimation - Getting Started

I understand the concept behind finding a Maximum Likelihood Estimator, but when I'm setting up the likelihood function, I'm having trouble understanding if I start with a summation to create the joint probability function, or if I start with a product.

I thought it depended on the original distribution - perhaps product for continuous random variables and summation for discrete. But then I saw in my notes we started with a product when finding the MLE for p in the binomial distribution, and used a summation for finding the MLE for $\theta$ in the exponential distribution.

Is there a guideline?

• Or is it always the product and the goal is to manipulate it into a summation, using natural logs and laws of exponents usually? – Carolyn Oct 25 '16 at 2:44
• You always start with the product - but due to some nice properties of exponents (especially for nice functions such as exponential family etc) you usually end up with a summation (assuming I understand your question). – Chinny84 Oct 25 '16 at 2:48
• @Chinny84 thank you! I thought a little more into it – Carolyn Oct 25 '16 at 2:53
• no worries! it is great you take the time to think about the technique instead of following it blindly - and in the real world this can cause a lot of problems. – Chinny84 Oct 25 '16 at 12:20
• It would be better to describe a simple case you're having trouble with and we'll try to assist you. – Royi Jul 28 '17 at 9:21

It makes little theoretical difference which you use, since logarithms are continuous strictly increasing functions. Taking the derivative of the product gives $\frac{d}{dx} \prod_i f_i(x) = \left(\sum_i \frac{f_i^\prime(x)}{f_i(x)}\right)\left( \prod_j f_j(x)\right)$ while the derivative of the sums of the logarithms gives $\frac{d}{dx} \sum_i \log_e(f_i(x)) = \sum_i \frac{f_i^\prime(x)}{f_i(x)}$, and these will have the same zeros if the likelihood is positive. If the individual likelihoods involve multiplication, powers and exponentiation, then their logarithms may sometimes be easier to handle