1
$\begingroup$

I have an error function $$err=\frac{1}{N}\left[\textbf{y}^T\ln{\textbf{x}}+(\textbf{1}-\textbf{y})^T\ln{(\textbf{1}-\textbf{x})}\right]$$ I need to find the gradient $\bigtriangledown_x{}err$, such that find derivation over vector $\textbf{x}$. I am learning matrix calculus and multivariable calculus and somehow I cannot figure out how to do it. I know what the derivative of $\ln{(x)}$ is but struggle to write it in vector notation e.g. $\frac{1}{\textbf{x}}$ ??? Which I believe is not right. I will be grateful for some help. Cheers!

$\endgroup$

1 Answer 1

1
$\begingroup$

The issue I think you are having is in thinking about what the log of a vector is. For me this is simply another vector function, i.e. for vector $\textbf{x} = [x_{1},\ldots,x_{n}]^{T}$

$\ln(1-\textbf{x}) = [\ln(1-x_{1}),\ldots,\ln(1-x_{n})]^{T}$

and so

$\displaystyle\nabla_{x}\ln(1-\textbf{x}) = \left[\frac{\text{d}}{\text{d}x_{1}}\ln(1-x_{1}), \ldots, \frac{\text{d}}{\text{d}x_{n}}\ln(1-x_{n})\right]^{T}$

Then we use the notation $\nabla_{x}\ln(\textbf{x})$ for the derivative of $\ln(\textbf{x})$ wrt the vector $\textbf{x}$. I.e.

$\nabla_{x}\text{err} = \displaystyle\frac{1}{N} [\textbf{y}^{T}\nabla_{x}\ln(\textbf{x}) + (1-\textbf{y})^{T}\nabla_{x}\ln(1-\textbf{x})]$.

This makes it clear what is being done, and the notation is not too verbose. As you mention $1/\textbf{x}$ does not make sense for a vector, so this notation would be wrong. Alternatively you could define a dummy variable $\textbf{z} = \nabla_{x}\ln(\textbf{x}) = [1/x_{1},\ldots,1/x_{n}]$ if you want the notation to be more compact.

$\endgroup$
2
  • $\begingroup$ Hi Keeran Brabazon. Thanks for your answer. I have probably not described my problem correctly. I knew what a derivative of each component of $\textbf{x}$ would be. I would end up with the same vector as yours (in brackets). The problem I have is the notation. There has to be something that let you express it as one symbol. Cheers! $\endgroup$
    – Celdor
    Oct 11, 2013 at 10:38
  • $\begingroup$ Thank you for your time. I hoped there could be something more visible and simpler in notation than that. Cheers! $\endgroup$
    – Celdor
    Oct 11, 2013 at 13:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .