I have to find the gradient of the following function:

$$ f(x) = \ln(1 + e^{-ab^Tx})$$

where $f: \mathbb{R}^n \rightarrow \mathbb{R}$, $b \in \mathbb{R}^n$, $a \in \mathbb{R}$ and $x \in \mathbb{R}^n$.

I'm attempting to apply the chain rule here but I'm finding it difficult to apply. The chain rule as I know it goes like:

for $h(x) = f(g(x))$

$$ \nabla h(x) = D f(x)^T \nabla g(f(x)) $$ where $f :\mathbb{R}^n \rightarrow \mathbb{R}^m$, $g: \mathbb{R}^m \rightarrow \mathbb{R}$ and $h : \mathbb{R}^n \rightarrow \mathbb{R}$

So I set, $g(x) = 1 + e^{-ab^Tx}$ and $f(y) = \ln(y)$. I'm not sure where to go with this now. My guess is that, $$Df(x)^T = \frac{1}{1+e^{-ab^Tx}}$$ and $$ \nabla g(f(x)) = -abe^{-ab^Tx}$$ So the final answer should be $$ \nabla f(x) = \frac{-abe^{-ab^Tx}}{e^{-ab^Tx}}$$

Is this correct? If wrong, any help to find the right answer would be appreciated. ${{}}$

  • 4
    $\begingroup$ You have written $x \in \mathbb R$. I think it should be $x \in \mathbb R^{n}$. $\endgroup$ – Kavi Rama Murthy Sep 18 '18 at 5:34
  • $\begingroup$ Welcome to math.SE! In addition to the important @KaviRamaMurthy's comment the expression "$f(x) = \ln(1 + e^{-ab^Tx})$" isn't a multivariable function, since only depends on $x$. So in that case you don't have to find a gradient, but the derivative. If not, please correct your grammar, so we can help you better. $\endgroup$ – manooooh Sep 18 '18 at 6:02
  • $\begingroup$ @KaviRamaMurthy corrected, thanks! $\endgroup$ – NewComer Sep 18 '18 at 11:29
  • $\begingroup$ @manooooh Since $x \in \mathbb{R}^n$ I thought it belongs to multi variable calculus. Isn't it? If not, what do you call these calculations? $\endgroup$ – NewComer Sep 18 '18 at 11:30
  • $\begingroup$ @manooooh Surely $b^T x \in \mathbb{R}^n \times \mathbb{R}^n$ $\endgroup$ – Kevin Sep 18 '18 at 11:33

I like to think of the chain rule with the notation

$$ D (f \circ g)(x) = Df(g(x)) \circ Dg(x) $$ (See the Wikipedia page on the chain rule. $Df(x)$ means the Jacobian matrix of $f$ at the point $x$)

I would decompose your function as $F(x) = f(g(x))$ where $g(x) = b^T x$ and $f(y) = \ln(1+e^{ay})$. Since $g(x) = b_1x_1 + \cdots b_n x_n$, then $Dg(x) = \nabla g(x) = b$. Also, $Df(y) = \frac{df}{dy} = \frac{ae^{ay}}{1+e^{ay}}$. So I think you get

$$ \nabla F(x) = \frac{ae^{ab^Tx}}{1+e^{ab^Tx}} \, b $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.