1
$\begingroup$

I have an error function $$err=\frac{1}{N}\left[\textbf{y}^T\ln{\textbf{x}}+(\textbf{1}-\textbf{y})^T\ln{(\textbf{1}-\textbf{x})}\right]$$ I need to find the gradient $\bigtriangledown_x{}err$, such that find derivation over vector $\textbf{x}$. I am learning matrix calculus and multivariable calculus and somehow I cannot figure out how to do it. I know what the derivative of $\ln{(x)}$ is but struggle to write it in vector notation e.g. $\frac{1}{\textbf{x}}$ ??? Which I believe is not right. I will be grateful for some help. Cheers!

$\endgroup$
1
$\begingroup$

The issue I think you are having is in thinking about what the log of a vector is. For me this is simply another vector function, i.e. for vector $\textbf{x} = [x_{1},\ldots,x_{n}]^{T}$

$\ln(1-\textbf{x}) = [\ln(1-x_{1}),\ldots,\ln(1-x_{n})]^{T}$

and so

$\displaystyle\nabla_{x}\ln(1-\textbf{x}) = \left[\frac{\text{d}}{\text{d}x_{1}}\ln(1-x_{1}), \ldots, \frac{\text{d}}{\text{d}x_{n}}\ln(1-x_{n})\right]^{T}$

Then we use the notation $\nabla_{x}\ln(\textbf{x})$ for the derivative of $\ln(\textbf{x})$ wrt the vector $\textbf{x}$. I.e.

$\nabla_{x}\text{err} = \displaystyle\frac{1}{N} [\textbf{y}^{T}\nabla_{x}\ln(\textbf{x}) + (1-\textbf{y})^{T}\nabla_{x}\ln(1-\textbf{x})]$.

This makes it clear what is being done, and the notation is not too verbose. As you mention $1/\textbf{x}$ does not make sense for a vector, so this notation would be wrong. Alternatively you could define a dummy variable $\textbf{z} = \nabla_{x}\ln(\textbf{x}) = [1/x_{1},\ldots,1/x_{n}]$ if you want the notation to be more compact.

$\endgroup$
  • $\begingroup$ Hi Keeran Brabazon. Thanks for your answer. I have probably not described my problem correctly. I knew what a derivative of each component of $\textbf{x}$ would be. I would end up with the same vector as yours (in brackets). The problem I have is the notation. There has to be something that let you express it as one symbol. Cheers! $\endgroup$ – Celdor Oct 11 '13 at 10:38
  • $\begingroup$ Thank you for your time. I hoped there could be something more visible and simpler in notation than that. Cheers! $\endgroup$ – Celdor Oct 11 '13 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.