Matrix derivative of square root Let A be an $n\times n$  matrix  (real valued), and set $S := \sqrt{A^t A}$. What is the matrix derivative of $S$ w.r.t. $A$? I.e., what is $\frac{\partial S}{\partial A}$?
(In case of invertibility issues, I might for the time being assume that $A^t A$ is positive definite, although for the applications I have in mind I might have only positive semi-definiteness). 
Thanks for help!
 A: If you are familiar with the vec operation for matrices, then you could proceed as follows.
$$\eqalign{
&S\,S 
  &= A^TA \cr
&S\,dS\,(I)+(I)\,dS\,S
  &= A^T\,dA\,(I)+(I)\,dA^T\,A \cr
&(I^T\otimes S+S^T\otimes I)\,{\rm vec}(dS)
  &= (I^T\otimes A^T)\,{\rm vec}(dA)+(A^T\otimes I)\,{\rm vec}(dA^T) \cr
&\Big(I\otimes S+S\otimes I\Big)\,{\rm vec}(dS)
  &= \Big((I\otimes A^T)+(A^T\otimes I)K\Big)\,{\rm vec}(dA) \cr
&\frac{\partial{\,\rm vec}(S)}{\partial{\,\rm vec}(A)}
  &= \Big(I\otimes S+S\otimes I\Big)^+
\Big((I\otimes A^T)+(A^T\otimes I)K\Big) \cr\cr
}$$
where $M^+$ denotes the pseudoinverse of $M$, $I$ is the identity matrix, and $K$ is the commutation matrix associated with the Kronecker product. The solution also takes advantage of the fact that $I$ and $S$ are symmetric.
A: Recall that you can perform a polar decomposition on $A$ such that:
$$A = QS$$
where $Q$ is an orthogonal matrix and $S$ is defined uniquely as precisely what you describe above when it exists. Hence, what you really want to calculate is:
$$\frac{\partial S}{\partial A} = \frac{\partial [Q^{-1}A]}{\partial A} = Q^{-1}\otimes\mathbf{I}$$
Note the result is a fourth-order linear transformation, since we are the describing the rate of change of a second-order object $S$ with respect to another second-order linear transformation $A$.
