3
$\begingroup$

I find it impossible to remember that the Jacobian of $f: \mathbb R^n \to \mathbb R^m$ is

$$ \begin{pmatrix} {\partial f_1 \over \partial x_1} & {\partial f_1 \over \partial x_2} & \dots & {\partial f_1 \over \partial x_n} \\ \vdots & \dots & \vdots & \vdots\\ {\partial f_m \over \partial x_1} & \dots & \dots & {\partial f_m \over \partial x_n} \end{pmatrix}$$

and not

$$ \begin{pmatrix} {\partial f_1 \over \partial x_1} & {\partial f_2 \over \partial x_1} & \dots & {\partial f_m \over \partial x_1} \\ \vdots & \dots & \vdots & \vdots\\ {\partial f_1 \over \partial x_n} & \dots & \dots & {\partial f_m \over \partial x_n} \end{pmatrix}$$

How to memorise this? Is there any reason why it's defined the first way and not the other?

$\endgroup$
3
  • $\begingroup$ for $f:\mathbb{R}^n\rightarrow\mathbb{R}$ the gradient is horizontal $\endgroup$
    – Set
    May 29, 2015 at 1:39
  • $\begingroup$ The reason it's horizontal is because vectors are vertical by default and thus you have $\nabla f(x_0)x$ as the linear approximation to $f$ at $x_0$ without having to take a transpose. $\endgroup$
    – Set
    May 29, 2015 at 1:44
  • $\begingroup$ @Tyroshipleasurebarge Thank you for your comment. $\endgroup$
    – a student
    May 29, 2015 at 2:09

3 Answers 3

5
$\begingroup$

Since $f : \mathbb{R}^n \to \mathbb{R}^m$, then the Jacobian is also a function $Jf : \mathbb{R}^n \to \mathbb{R}^m$. This means that we will multiply the matrix $Jf$ by a $n \times 1$ column vector. Now a matrix product is only defined if the number of columns of the lefthand matrix is the same as the number of rows of the righthand matrix, so we must have $$ (m \times \underbrace{n) \times (n}_{=} \times 1) $$ which results in an $m \times 1$ column vector in $\mathbb{R}^m$. Thus $Jf$ must be $m \times n$, hence has $m$ rows and $n$ columns, as in your first formula.

$\endgroup$
5
$\begingroup$

Consider a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$. You of course have the gradient:

$$\nabla f= \left( \frac{\partial f}{\partial x_1},..., \frac{\partial f}{\partial x_n} \right)$$

If you have a function $f: \mathbb{R}^n \rightarrow \mathbb{R} ^m$, you have $m$ functions like above. Just stack the gradients in the matrix.

$\begin{pmatrix} \nabla f_1 \\ \nabla f_2 \\ ... \\ \nabla f_m\end{pmatrix}$

You could also remember that the jacobian must take a $n$-vector to a $m$-vector, hence it must be a $m \times n$ matrix.

$\endgroup$
2
  • 1
    $\begingroup$ I didn't know, I thought the gradient was vertical. $\endgroup$
    – a student
    May 29, 2015 at 2:10
  • 3
    $\begingroup$ Looking at Wikipedia it seems that the gradient is vertical: The partial derivatives are the coefficients of a vector and vectors are vertical. Right? $\endgroup$
    – a student
    May 29, 2015 at 2:11
1
$\begingroup$

The Jacobian at $x$, $J(f)_x$ is the matrix that makes $$f(x+h) = f(x) + J(f)_x h + o(\|h\|)$$ free of any transposition. With only column vectors for $x$ and $h$ and no transpose for the Jacobian, this is the only possibility. Then I work backwards to figure out the convention that is indeed impossible to remember.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .