Remembering the definition of the Jacobian: any tips? I find it impossible to remember that the Jacobian  of $f: \mathbb R^n \to \mathbb R^m$ is
$$ \begin{pmatrix}
{\partial f_1 \over \partial x_1} & {\partial f_1 \over \partial x_2} & \dots & {\partial f_1 \over \partial x_n} \\
\vdots & \dots & \vdots & \vdots\\
{\partial f_m \over \partial x_1} & \dots & \dots & {\partial f_m \over \partial x_n}
\end{pmatrix}$$
and not
$$ \begin{pmatrix}
{\partial f_1 \over \partial x_1} & {\partial f_2 \over \partial x_1} & \dots & {\partial f_m \over \partial x_1} \\
\vdots & \dots & \vdots & \vdots\\
{\partial f_1 \over \partial x_n} & \dots & \dots & {\partial f_m \over \partial x_n}
\end{pmatrix}$$ 

How to memorise this? Is there any reason why it's defined the first
  way and not the other?

 A: Consider a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$. You of course have the gradient:
$$\nabla f= \left( \frac{\partial f}{\partial x_1},..., \frac{\partial f}{\partial x_n} \right)$$
If you have a function $f: \mathbb{R}^n \rightarrow \mathbb{R} ^m$, you have $m$ functions like above. Just stack the gradients in the matrix.
$\begin{pmatrix} \nabla f_1 \\ \nabla f_2 \\ ... \\ \nabla f_m\end{pmatrix}$
You could also remember that the jacobian must take a $n$-vector to a $m$-vector, hence it must be a $m \times n$ matrix.
A: Since $f : \mathbb{R}^n \to \mathbb{R}^m$, then the Jacobian is also a function $Jf : \mathbb{R}^n \to \mathbb{R}^m$.  This means that we will multiply the matrix $Jf$ by a $n \times 1$ column vector.  Now a matrix product is only defined if the number of columns of the lefthand matrix is the same as the number of rows of the righthand matrix, so we must have
$$
(m \times \underbrace{n) \times (n}_{=} \times 1)
$$
which results in an $m \times 1$ column vector in $\mathbb{R}^m$.  Thus $Jf$ must be $m \times n$, hence has $m$ rows and $n$ columns, as in your first formula.
A: The Jacobian at $x$, $J(f)_x$ is the matrix that makes
$$f(x+h) = f(x) + J(f)_x h + o(\|h\|)$$
free of any transposition. Then I work backwards to figure out the convention that is indeed impossible to remember.
