My question is: Given a matrix $A$ and its eigenvector $v$ which corresponds to $A$'s maximum eigenvalue, is there a closed form formula to calculate the derivative

$$\frac{\partial(u^Tv)}{\partial A}$$

where $u$ is an unrelated vector?

Please help! Thanks in advance!


I check matrix cookbook which shows

$$\partial v = (\lambda I-A)^\dagger\partial(A) v$$

where $\lambda$ is the corresponding eigenvalue, $\dagger$ is the symbol of pseudo-inverse. But I still don't know how to calculate the desired derivative.


Start with the eigenvalue equation, and take differentials $$ \eqalign { 0 &= (A-\lambda I)v \cr &= dA\,v + (A-\lambda I)dv \cr dA\,v &= (\lambda I-A)dv } $$ For notational convenience, let $M=(\lambda I-A)^{\dagger}$.

Then proceeding to the least squares solution, we obtain the cookbook result $$ \eqalign { dv &= M\,dA\,v \cr } $$ From there, we can pre-multiply by $u^T$ to obtain $$ \eqalign { u^Tdv &= u^TM\,dA\,v \cr d(u^Tv) &= p^TdA\,v \cr &= pv^T:dA \cr } $$ where $p=M^Tu\,\,$ and the colon represents the Frobenius product, $\,X\!:\!Y=tr(X^TY)$.

Since $df = (\frac{\partial f}{\partial A}):dA$, the derivative must be $$ \eqalign { \frac{\partial\,(u^Tv)}{\partial A} &= pv^T \cr &= M^{T}uv^T \cr &= (\lambda I-A^T)^{\dagger} \, uv^T } $$

  • $\begingroup$ Thanks, @greg ! That's what I want. I forget to use the definition of differential. $\endgroup$ – fetcher May 31 '15 at 3:35
  • 1
    $\begingroup$ Please be aware that the cookbook result comes with all sorts of restrictions on $A$, i.e. it must be real, symmetric, with distinct eigenvalues. Also the eigenvector must be normalized, $\|v\|=1$. $\endgroup$ – lynn May 31 '15 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.