# Jacobian of a function mapping vectors to vectors

I believe the problem of trying to find the Jacobian of the following function highlights a lack of understanding of some concept on my part. I was hoping someone could either provide specific advice about solving this problem, or computing Jacobians in general.

Consider the mapping $h : \mathbb{R}^n \rightarrow \mathbb{R}^n$ where the domain is length-$n$ column vectors and the range length-$n$ row vectors (or a transposed vector, if you like). The function is $$h(x) = \frac{\eta v' + (M x)'}{(\eta + u'x)^2},$$ where the constants $v$ and $u$ are (column) vectors, $\eta$ is a scalar, and $M$ is a square matrix.

So far as I know, the quotient rule for vectors is $$\nabla\left(\frac{f}{g}\right) = \frac{g\nabla f - f \nabla g}{g^2}$$ and \begin{align*} \nabla f &= M'\\ \nabla g &= 2(\eta + u'x) u' \end{align*} Putting it all together, I get $$\nabla h = \frac{(\eta + u'x)^2 M' - [\eta v' + (M x)'] 2(\eta + u'x) u'}{(\eta + u'x)^4}.$$ This expression is clearly not right, and to see why evaluate the Jacobian at $x = \mathbf{0}$: $$\nabla h(0) = \frac{\eta^2 M' - 2\eta^2 v' u'}{\eta^4}$$ The resulting expression should be a $n \times n$ matrix, but in the second term we have two (row) vectors multiplied by one another. It seems likely there should be some sort of outer product here, but I'm not sure where my math is going wrong.

You also need to think harder about matrix multiplication and transposes. You implicitly used the relation that for $f(x) = (Mx)^T$, $\nabla f = M^T$, which is incorrect. Recall that $\nabla_y f(x) = \lim_{h\to 0} \frac1h (f(x + hy) - f(x))$, where $y$ is a vector, you see that $\nabla_yf = (My)^T$. Key is the fact that $(Mx)^T \neq M^Tx$, but $(Mx)^T = x^T M^T$. In fact, why do you insist on having the output of the function $h$ be a row vector? This I think is the spot that is really tripping you up. – Willie Wong Jan 19 '11 at 22:41
My suggestion for minimizing notational confusion is to focus on directional derivative, in the direction of some fixed vector $w$. This derivative $D_w h$ is always a function of the same nature as $h$: here, it will also eat column vectors and spit out row vectors. For the numerator the computation is easy since it's linear: $$D_w(\eta v' + (M x)') = (Mw)' \tag1$$ while $(\eta + u'x)^{-2}$ has, by the chain rule, $$D_w((\eta + u'x)^{-2}) = -2 (\eta + u'x)^{-3} D_w(\eta + u'x) =-2 (\eta + u'x)^{-3} (u'w) \tag2$$ Combine (1) and (2) by the product rule: $$D_w h = -2 (\eta + u'x)^{-3} (u'w) (\eta v' + (M x)') +(\eta + u'x)^{-2}(Mw)' \tag3$$ As promised, the right hand side of (3) is a row vector. It depends linearly on $w$ and thus defines a $(1,1)$ tensor field.