2
$\begingroup$

How to compute the gradient of \begin{align} L\left(W, \gamma, \beta\right) := -y^T \log \left(f \left(W x ; \gamma, \beta \right) \right) \end{align} with respect to $\left\{W, \gamma, \beta\right\}$, where $x \in \mathbb{R}^n$, $W \in \mathbb{R}^{m \times n}$, and $y \in \mathbb{R}^m$, but $y_i \in \{0,1\}$, $f(z; \gamma, \beta)$ is parameterized by $\gamma$ and $\beta$?

The definition of
$$\eqalign{ f(z; \gamma, \beta) &= \gamma \ \left( z- \mu(z) \right) \ \left( \sigma(z) + \epsilon \right)^{-1/2} + \beta \cr \mu(z) &= \alpha \ 1^Tz \cr \sigma(z) &= \alpha \sum_{k=1}^m \left( z[k] - \mu(z) \right)^2 \equiv \alpha 1^T \left[ \left( z- \mu(z) \right) \odot \left(z - \mu(z) \right) \right]\cr }$$ where $1^T$ is a row vector with all ones, $\odot$ is an element-wise multiplication, and $\alpha$ and $\epsilon$ are known scalars.

Thank you so much in advance for your help

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.