Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm taking a course on machine learning and came across a model answer which I can't wrap my head around.

The whole model answer is the 2nd exercise here:

The part I'm specifically having trouble understanding is the last 2 lines of the 2nd page, ie this:

How does the left side of the upper equation become just the sum of indicator variables? How does the right side simplify to just lambda*p_i?

Any help is appreciated.

share|cite|improve this question
up vote 2 down vote accepted

I say there be typos. Particular the suddenly appearing subscript $k$ does not make sense.

Straightforward differentiation gives: $${\partial \over {\partial p_i}} L(p|X)={1 \over p_i}\sum_{t=1}^N x^t_i + \lambda$$ Setting this equal to zero gives us: $$\sum_{t=1}^N x^t_i + \lambda p_i=0$$ Proceeding from which and reading the subscript $k$ in your text as $i$ leads to the same ultimate answer.

share|cite|improve this answer
Thanks for the answer, but it's still a bit unclear to me how differentiating with respect to p_i, (sum(i=1...K)(p_i) equals one? – tsiki Oct 15 '10 at 12:41
$\partial \log p_j/\partial p_i=0$ when $j \neq i$, so all terms other than the i-th disappear. – Jyotirmoy Bhattacharya Oct 15 '10 at 12:46
Ahh now I see it, thank you. – tsiki Oct 15 '10 at 12:51

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.