Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Suppose $J(\mathbf{A})$ is defined as follows $$J=\text{tr}(\log \mathbf{P})$$ $$\mathbf{P}=\frac{e^\mathbf{A}}{\mathbf{1} \mathbf{1}' e^\mathbf{A}}$$

where division, exp and log are taken pointwise, $\mathbf{1}$ is a column vector of ones and $\mathbf{A}$ is square. What's the easiest way of showing that $\mathbf{I}-\mathbf{P}$ is the gradient of $J(\mathbf{A})$?

share|improve this question
    
Gradient with respect to what? –  martini Nov 30 '12 at 7:39
    
Gradient with respect to A, added clarification –  Yaroslav Bulatov Nov 30 '12 at 7:46

1 Answer 1

up vote 1 down vote accepted

$\def\tr{\operatorname{tr}}$Just computing, I'd say. We have wrting $P = (p_{ij})$, $A = (a_{ij})$ and $d$ for the dimension \begin{align*} p_{ij} &= \frac{\exp a_{ij}}{\sum_{k=1}^d \exp a_{kj}}\\ \log p_{ij} &= a_{ij} - \log \sum_{k=1}^d \exp a_{kj}\\ J(A) &= \tr\log P\\ &= \tr A - \sum_{l=1}^d \log \sum_{k=1}^d \exp a_{kl} \end{align*} So we have \begin{align*} \partial_{a_{ii}}J(A) &= 1 - \frac 1{\sum_{k=1}^d \exp a_{ki}}\cdot \exp a_{ii}\\ &= 1 - p_{ii}\\ \text{and for $i\ne j$:}\\ \partial_{a_{ij}} &= -\frac 1{\sum_{k=1}^d \exp a_{kj}}\cdot \exp a_{ij}\\ &= -p_{ij} \end{align*} So we have $\nabla J(A) = \mathrm{id} - P(A)$

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.