Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I believe the problem of trying to find the Jacobian of the following function highlights a lack of understanding of some concept on my part. I was hoping someone could either provide specific advice about solving this problem, or computing Jacobians in general.

Consider the mapping $h : \mathbb{R}^n \rightarrow \mathbb{R}^n$ where the domain is length-$n$ column vectors and the range length-$n$ row vectors (or a transposed vector, if you like). The function is $$h(x) = \frac{\eta v' + (M x)'}{(\eta + u'x)^2},$$ where the constants $v$ and $u$ are (column) vectors, $\eta$ is a scalar, and $M$ is a square matrix.

So far as I know, the quotient rule for vectors is $$\nabla\left(\frac{f}{g}\right) = \frac{g\nabla f - f \nabla g}{g^2}$$ and \begin{align*} \nabla f &= M'\\ \nabla g &= 2(\eta + u'x) u' \end{align*} Putting it all together, I get $$\nabla h = \frac{(\eta + u'x)^2 M' - [\eta v' + (M x)'] 2(\eta + u'x) u'}{(\eta + u'x)^4}.$$ This expression is clearly not right, and to see why evaluate the Jacobian at $x = \mathbf{0}$: $$\nabla h(0) = \frac{\eta^2 M' - 2\eta^2 v' u'}{\eta^4}$$ The resulting expression should be a $n \times n$ matrix, but in the second term we have two (row) vectors multiplied by one another. It seems likely there should be some sort of outer product here, but I'm not sure where my math is going wrong.

Any help you can provide is greatly appreciated.

share|cite|improve this question
What is a vector quotient? – Rasmus Jan 19 '11 at 21:26
Gradients are only defined for functions mapping vectors to scalars. The object you're looking for is called a Jacobian: – Qiaochu Yuan Jan 19 '11 at 21:27
@Rasmus I simply meant the quotient rule when taking derivatives with respect to a vector. – RandomGuy Jan 19 '11 at 22:25
@Qiaochu Yuan: My apologies, I got confused with nomenclature as this expression was arrived at by computing a gradient. I'm now trying to apply the same technique to this expression to compute the Hessian. Does the fact that I am computing essentially a vector of gradients (thus the matrix) imply the technique to compute it should be different? – RandomGuy Jan 19 '11 at 22:29
You also need to think harder about matrix multiplication and transposes. You implicitly used the relation that for $f(x) = (Mx)^T$, $\nabla f = M^T$, which is incorrect. Recall that $\nabla_y f(x) = \lim_{h\to 0} \frac1h (f(x + hy) - f(x))$, where $y$ is a vector, you see that $\nabla_yf = (My)^T$. Key is the fact that $(Mx)^T \neq M^Tx$, but $(Mx)^T = x^T M^T$. In fact, why do you insist on having the output of the function $h$ be a row vector? This I think is the spot that is really tripping you up. – Willie Wong Jan 19 '11 at 22:41

My suggestion for minimizing notational confusion is to focus on directional derivative, in the direction of some fixed vector $w$. This derivative $D_w h$ is always a function of the same nature as $h$: here, it will also eat column vectors and spit out row vectors. For the numerator the computation is easy since it's linear: $$D_w(\eta v' + (M x)') = (Mw)' \tag1$$ while $(\eta + u'x)^{-2}$ has, by the chain rule, $$D_w((\eta + u'x)^{-2}) = -2 (\eta + u'x)^{-3} D_w(\eta + u'x) =-2 (\eta + u'x)^{-3} (u'w) \tag2$$ Combine (1) and (2) by the product rule: $$D_w h = -2 (\eta + u'x)^{-3} (u'w) (\eta v' + (M x)') +(\eta + u'x)^{-2}(Mw)' \tag3 $$ As promised, the right hand side of (3) is a row vector. It depends linearly on $w$ and thus defines a $(1,1)$ tensor field.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.