Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am working with the probability likelihood function $$ \log \prod\limits_{i=1}^{n} x_i^{y_i} + \log \prod\limits_{i=1}^{n}\left(1-{{x}_{i}}\right)^{n_i-y_i}. $$
I want to take the derivative with respect to $x_i$.

How should I do it? Thanks in advance.

share|improve this question
    
You'll need the chain rule and a memory of logarithm identities, e.g. $\log(a^b c^d)=b\log\,a+d\log\,c$... –  J. M. May 2 '12 at 17:57
    
how do I take derivative of a product? –  edwin May 2 '12 at 18:05
    
What you actually have is the logarithm of a product, so you can use that identity I mentioned earlier before differentiating... –  J. M. May 2 '12 at 18:06
    
are you saying: $\log \prod\limits_{i=n}^{n}{{{y}_{i}}}\prod\limits_{i=n}^{n}{{{x}_{i}}}+\log \prod\limits_{i=1}^{n}{({{n}_{i}}-{{y}_{i}})}\prod\limits_{i=1}^{n}{(1-{{x}_{i}}‌​)}$ –  edwin May 2 '12 at 18:09
    
$=\log \prod\limits_{i=n}^{n}{{{y}_{i}}}+\prod\limits_{i=n}^{n}{{{x}_{i}}}+\log \prod\limits_{i=1}^{n}{({{n}_{i}}-{{y}_{i}})}+\prod\limits_{i}^{n}{(1-{{x}_{i}})‌​}$ @J.M. –  edwin May 2 '12 at 18:14

2 Answers 2

$$\log \prod\limits_{i=1}^{n} x_i^{y_i} + \log \prod\limits_{i=1}^{n}\left(1-{{x}_{i}}\right)^{n_i-y_i} =\sum_{i=1}^{n} {y_i}\log x_i + \sum\limits_{i=1}^{n} ({n_i-y_i})\log \left(1-{{x}_{i}}\right)$$

so the derivative you are looking for is $\dfrac{y_i}{x_i} - \dfrac{n_i-y_i}{1-x_i}$. This will be zero when $x_i=\frac{y_i}{n_i}$ which probably has an intuitive interpretation.

share|improve this answer
    
There seems to be a stray product on the right-hand side... –  J. M. May 2 '12 at 19:04
    
@J.M.:indeed - thank you –  Henry May 2 '12 at 20:15

Does this look right?
$\log \prod\limits_{i=1}^{m}{\left( \begin{align} & {{n}_{i}} \\ & {{y}_{i}} \\ \end{align} \right)\theta _{i}^{{{y}_{i}}}{{(1-{{\theta }_{i}})}^{{{n}_{i}}-{{y}_{i}}}}}$
$=\log \prod\limits_{i=n}^{n}{{{\theta }_{i}}^{{{y}_{i}}}}+\log \prod\limits_{i}^{n}{{{(1-{{\theta }_{i}})}^{{{n}_{i}}-{{y}_{i}}}}}$ $\text{Note: I ignored the therm } \left( \begin{align} & {{n}_{i}} \\ & {{y}_{i}} \\ \end{align} \right)$ $=\sum\limits_{i=1}^{m}{{{y}_{i}}\log {{\theta }_{i}}}+\sum\limits_{i=1}^{m}{({{n}_{i}}-{{y}_{i}})\log (1-{{\theta }_{i}})}$

Calculating first derivative:
$$\frac{\partial }{\partial \theta }=\frac{\sum{{{y}_{i}}}}{{{\theta }_{i}}}-\frac{\sum{({{n}_{i}}-{{y}_{i}})}}{1-{{\theta }_{i}}}=0$$ $$\frac{\sum{{{y}_{i}}}}{{{\theta }_{i}}}=\frac{\sum{({{n}_{i}}-{{y}_{i}})}}{1-{{\theta }_{i}}}$$ $$\sum{{{y}_{i}}-{{\theta }_{i}}\sum{{{y}_{i}}={{\theta }_{i}}\sum{({{n}_{i}}-{{y}_{i}})}}}$$ $$\sum{{{y}_{i}}={{\theta }_{i}}\left( \sum{({{n}_{i}}-{{y}_{i}})+\sum{{{y}_{i}}}} \right)}$$ $$\sum{{{y}_{i}}={{\theta }_{i}}(\sum{{{n}_{i}}})}$$ $${{\hat{\theta }}_{i}}=\frac{\sum{{{y}_{i}}}}{\sum{{{n}_{i}}}}$$

share|improve this answer
    
this is wrong! We are taking derivative w.r.t. each ${\theta}_{i}$ –  user1061210 May 2 '12 at 19:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.