# Jacobian of a function mapping vectors to vectors

I believe the problem of trying to find the Jacobian of the following function highlights a lack of understanding of some concept on my part. I was hoping someone could either provide specific advice about solving this problem, or computing Jacobians in general.

Consider the mapping $h : \mathbb{R}^n \rightarrow \mathbb{R}^n$ where the domain is length-$n$ column vectors and the range length-$n$ row vectors (or a transposed vector, if you like). The function is $$h(x) = \frac{\eta v' + (M x)'}{(\eta + u'x)^2},$$ where the constants $v$ and $u$ are (column) vectors, $\eta$ is a scalar, and $M$ is a square matrix.

So far as I know, the quotient rule for vectors is $$\nabla\left(\frac{f}{g}\right) = \frac{g\nabla f - f \nabla g}{g^2}$$ and \begin{align*} \nabla f &= M'\\ \nabla g &= 2(\eta + u'x) u' \end{align*} Putting it all together, I get $$\nabla h = \frac{(\eta + u'x)^2 M' - [\eta v' + (M x)'] 2(\eta + u'x) u'}{(\eta + u'x)^4}.$$ This expression is clearly not right, and to see why evaluate the Jacobian at $x = \mathbf{0}$: $$\nabla h(0) = \frac{\eta^2 M' - 2\eta^2 v' u'}{\eta^4}$$ The resulting expression should be a $n \times n$ matrix, but in the second term we have two (row) vectors multiplied by one another. It seems likely there should be some sort of outer product here, but I'm not sure where my math is going wrong.

Any help you can provide is greatly appreciated.

-
What is a vector quotient? –  Rasmus Jan 19 '11 at 21:26
Gradients are only defined for functions mapping vectors to scalars. The object you're looking for is called a Jacobian: en.wikipedia.org/wiki/Jacobian_matrix_and_determinant –  Qiaochu Yuan Jan 19 '11 at 21:27
@Rasmus I simply meant the quotient rule when taking derivatives with respect to a vector. –  RandomGuy Jan 19 '11 at 22:25
@Qiaochu Yuan: My apologies, I got confused with nomenclature as this expression was arrived at by computing a gradient. I'm now trying to apply the same technique to this expression to compute the Hessian. Does the fact that I am computing essentially a vector of gradients (thus the matrix) imply the technique to compute it should be different? –  RandomGuy Jan 19 '11 at 22:29
You also need to think harder about matrix multiplication and transposes. You implicitly used the relation that for $f(x) = (Mx)^T$, $\nabla f = M^T$, which is incorrect. Recall that $\nabla_y f(x) = \lim_{h\to 0} \frac1h (f(x + hy) - f(x))$, where $y$ is a vector, you see that $\nabla_yf = (My)^T$. Key is the fact that $(Mx)^T \neq M^Tx$, but $(Mx)^T = x^T M^T$. In fact, why do you insist on having the output of the function $h$ be a row vector? This I think is the spot that is really tripping you up. –  Willie Wong Jan 19 '11 at 22:41
show 6 more comments

My suggestion for minimizing notational confusion is to focus on directional derivative, in the direction of some fixed vector $w$. This derivative $D_w h$ is always a function of the same nature as $h$: here, it will also eat column vectors and spit out row vectors. For the numerator the computation is easy since it's linear: $$D_w(\eta v' + (M x)') = (Mw)' \tag1$$ while $(\eta + u'x)^{-2}$ has, by the chain rule, $$D_w((\eta + u'x)^{-2}) = -2 (\eta + u'x)^{-3} D_w(\eta + u'x) =-2 (\eta + u'x)^{-3} (u'w) \tag2$$ Combine (1) and (2) by the product rule: $$D_w h = -2 (\eta + u'x)^{-3} (u'w) (\eta v' + (M x)') +(\eta + u'x)^{-2}(Mw)' \tag3$$ As promised, the right hand side of (3) is a row vector. It depends linearly on $w$ and thus defines a $(1,1)$ tensor field.