# Are these two gradient notations equivalent? ($\nabla f(x_n) = \frac{\partial}{\partial x_n} f$)

In gradient descent algorithms (especially when talking about multivariate regression), when they are talking about the gradient at a given point, sometimes I find the notation: $\nabla f(x_n)$ and sometimes it is the notation: $\frac{\partial}{\partial x_n} f$

For example:

• Wikipedia says: $x_{n+1} = x_n - \alpha \nabla f(x_n)$
• Andrew Ng (on Coursera) says: $\theta_j = \theta_j - \alpha \frac{\partial}{\partial \theta_j} J (\theta)$

So I was wondering if the two are the same, or if there differences when we should use one notation over the other.

• In a Hilbert space, there is a correspondence between linear functionals and points in the space. The gradient is the point corresponding to the linear functional $h \mapsto {\partial f(x) \over \partial x} h$. – copper.hat Feb 13 '17 at 3:32
• I'm just starting this, so for someone who doesn't know (yet) what a Hilbet space is, can I consider that these two notations are the same? – Ryan B. Feb 13 '17 at 3:33
• If you are working in $\mathbb{R}^n$, then think of ${\partial f(x) \over \partial x}$ as a row vector and $\nabla f(x) = {\partial f(x) \over \partial x}^T$ as a column vector. – copper.hat Feb 13 '17 at 3:35
• Thank you very much, it's way clearer now (I've spent hours on this). – Ryan B. Feb 13 '17 at 3:42
• They contain exactly the same information in $\mathbb{R}^n$. – copper.hat Feb 13 '17 at 3:52

$\nabla f(x_n)$ does NOT mean $\partial f/\partial x_n$. In the context of the wikipedia article on gradient descent, $x_n$ is a just a point in (say) $\mathbb{R}^3$. For example $x_n$ could be the point (1,0,2).
$\nabla f(x_n)$ is the vector $(\frac{\partial f}{\partial x} (x_n),\frac{\partial f}{\partial y} (x_n),\frac{\partial f}{\partial z} (x_n))$.
$\partial f/\partial x_n$ implies that you are labelling the coordinates $x_1, x_2,$ etc. Maybe $x_n$ happens to correspond to the coordinate $z$ in the 3d case, then this would be $\partial f/\partial z$ (which you would evaluate at some particular point).
• Follow up question: Say $x$ is a vector. then what is the difference between $\frac{\partial f}{\partial x}$ and $\nabla_x f$? Does this change anything? – Toke Faurby May 18 '17 at 0:20