Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose $J(\mathbf{A})$ is defined as follows $$J=\text{tr}(\log \mathbf{P})$$ $$\mathbf{P}=\frac{e^\mathbf{A}}{\mathbf{1} \mathbf{1}' e^\mathbf{A}}$$

where division, exp and log are taken pointwise, $\mathbf{1}$ is a column vector of ones and $\mathbf{A}$ is square. What's the easiest way of showing that $\mathbf{I}-\mathbf{P}$ is the gradient of $J(\mathbf{A})$?

share|cite|improve this question
Gradient with respect to what? – martini Nov 30 '12 at 7:39
Gradient with respect to A, added clarification – Yaroslav Bulatov Nov 30 '12 at 7:46
up vote 1 down vote accepted

$\def\tr{\operatorname{tr}}$Just computing, I'd say. We have wrting $P = (p_{ij})$, $A = (a_{ij})$ and $d$ for the dimension \begin{align*} p_{ij} &= \frac{\exp a_{ij}}{\sum_{k=1}^d \exp a_{kj}}\\ \log p_{ij} &= a_{ij} - \log \sum_{k=1}^d \exp a_{kj}\\ J(A) &= \tr\log P\\ &= \tr A - \sum_{l=1}^d \log \sum_{k=1}^d \exp a_{kl} \end{align*} So we have \begin{align*} \partial_{a_{ii}}J(A) &= 1 - \frac 1{\sum_{k=1}^d \exp a_{ki}}\cdot \exp a_{ii}\\ &= 1 - p_{ii}\\ \text{and for $i\ne j$:}\\ \partial_{a_{ij}} &= -\frac 1{\sum_{k=1}^d \exp a_{kj}}\cdot \exp a_{ij}\\ &= -p_{ij} \end{align*} So we have $\nabla J(A) = \mathrm{id} - P(A)$

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.