0
$\begingroup$

Given $\mathbf { f } _ { t } = \sigma \left( \mathbf { W } _ { f } \left[ \mathbf { h } _ { t - 1 } , \mathbf { x } _ { t } \right] ^ { T } + \mathbf { b } _ { f } \right)$, where $\sigma$ is the sigmoid function.

Then, $\frac { d \mathbf { f } _ { t } } { d w } = \frac { \partial \mathbf { f } _ { t } } { \partial w } + \frac { \partial \mathbf { f } _ { t } } { \partial \mathbf { h } _ { t - 1 } } \frac { d \mathbf { h } _ { t - 1 } } { d w }$.

But, how to understand $diag$ in
$\frac { \partial \mathbf { f } _ { t } } { \partial w } = \operatorname { diag } \left[ \mathbf { f } _ { t } \odot \left( 1 - \mathbf { f } _ { t } \right) \right] \cdot \mathbf { E } ^ { f } ( w ) \cdot \left[ \mathbf { h } _ { t - 1 } , \mathbf { x } _ { t } \right] ^ { T }$ ?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.