How to compute the gradient of \begin{align} L\left(W, \gamma, \beta\right) := -y^T \log \left(f \left(W x ; \gamma, \beta \right) \right) \end{align} with respect to $\left\{W, \gamma, \beta\right\}$, where $x \in \mathbb{R}^n$, $W \in \mathbb{R}^{m \times n}$, and $y \in \mathbb{R}^m$, but $y_i \in \{0,1\}$, $f(z; \gamma, \beta)$ is parameterized by $\gamma$ and $\beta$?

The definition of
$$\eqalign{ f(z; \gamma, \beta) &= \gamma \ \left( z- \mu(z) \right) \ \left( \sigma(z) + \epsilon \right)^{-1/2} + \beta \cr \mu(z) &= \alpha \ 1^Tz \cr \sigma(z) &= \alpha \sum_{k=1}^m \left( z[k] - \mu(z) \right)^2 \equiv \alpha 1^T \left[ \left( z- \mu(z) \right) \odot \left(z - \mu(z) \right) \right]\cr }$$ where $1^T$ is a row vector with all ones, $\odot$ is an element-wise multiplication, and $\alpha$ and $\epsilon$ are known scalars.

Thank you so much in advance for your help


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.