According to this article (See: the section "Derivatives of vector element-wise binary operators"), Jacobian Matrix is nothing but a Diagonal Identity Matrix.

I am failing to understand What is so special about Jacobian Matrix then? Why don't we talk about an Identity matrix then?

When is a Jacobian Matrix not diagonal?

  • $\begingroup$ your article does not say that. $\endgroup$ – dezdichado Jan 1 at 9:31
  • $\begingroup$ @dezdichado, see "Derivatives of vector element-wise binary operators". $\endgroup$ – user366312 Jan 1 at 9:41

From the article,

That's quite a furball, but fortunately the Jacobian is very often a diagonal matrix, a matrix that is zero everywhere but the diagonal.

is just an empirical claim that most of the Jacobians that you meet in this domain (deep learning according to this article) are diagonal. It is not true in general.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.