Derivative of eigenvalue equation dependent on vector parameters For an eigenpair $(\lambda,v)$ we have $Av=\lambda v$. Assume that both $A \in \mathbb{R}^{n \times n}$ and $v \in \mathbb{R}^n$ depend on $n$ parameters represented as a vector $p=[p1,p2,\dots,p_n]$. I want to know how the eigenvalue $\lambda$ changes with respect to $p$, but I am not sure if I am differentiating $A(p)v(p) = \lambda(p) v(p)$ correctly.
Let $y = A(p)v(p)$ and $z = \lambda(p)v(p)$. Then,
\begin{align}
\frac{\partial y}{\partial p} &= \frac{\partial y}{\partial p}
\\[1em] \frac{\partial y}{\partial A} \frac{\partial A}{\partial p} v + A \frac{\partial y}{\partial v} \frac{\partial v}{\partial p} &= \frac{\partial z}{\partial \lambda} \frac{\partial \lambda}{\partial p} v + \lambda \frac{\partial z}{\partial v} \frac{\partial v}{\partial p}\\[1em]
\frac{\partial A}{\partial p}(v,v) + A^2 \frac{\partial v}{\partial p} 
 &= \bigg(v^T \frac{\partial \lambda}{\partial p}\bigg) v + \lambda^2 \frac{\partial v}{\partial p} 
\end{align}
Premultiplying both sides by the left eigenvector $w^T$ (i.e., $w^TA=w^T\lambda)$, and using $w^Tv=1$ to simplify, I get
$$
\begin{align}
w^T \frac{\partial A}{\partial p}(v,v)
 &= v^T \frac{\partial \lambda}{\partial p}
\end{align}
$$
I am just not sure about the step where I go from $\frac{\partial y}{\partial A} \frac{\partial A}{\partial p} v = \frac{\partial A}{\partial p}(v,v)$ since $\frac{\partial A}{\partial p}$ is a third order tensor.
 A: We can do this fairly easily just by differentiating the eigenvalue equation $Av = \lambda v$,
$$
A' v + A v' = \lambda' v + \lambda v'
$$
Now multiply through on the left by $w^T$, where $w$ is the left eigenvector for the eigenvalue $\lambda$ (that is, $w^TA = \lambda w^T$, and also $w^Tv = 1$). Then we have
$$
w^TA'v + \lambda w^Tv' = \lambda'w^T v + \lambda w^T v' \Longrightarrow \lambda' = w^T A' v
$$
If you have multiple parameters, then for each parameter $p$, 
$$
\frac{\partial\lambda}{\partial p} = w^T\frac{\partial A}{\partial p} v
$$
If you've done quantum mechanics before, you'll recognize this as essentially time-independent perturbation theory.

Now, there's an important thing to watch out for here. Consider the derivative of the $i$th eigenvector $v'_i$. An abbreviated derivation of the expression for it is
\begin{multline}
A' v_i + A v_i' = \lambda_i' v_i + \lambda_i v_i'\Longrightarrow \sum_{k=1}^n v_kw_k^TA' v_i + A v_i' =  v_iw_i^T A' v_i + \lambda_i v_i'\\\Longrightarrow (A-\lambda_i)v_i' = \sum_{k\ne i}^n v_kw_k^TA' v_i \Longrightarrow \sum_{k\ne i}^n(\lambda_k-\lambda_i) v_kw_k^Tv_i' = \sum_{k\ne i}^n v_kw_k^TA' v_i\\\Longrightarrow v_i' =  \sum_{k\ne i}^n v_k\frac{w_k^TA' v_i}{\lambda_k-\lambda_i} + c_iv_i,
\end{multline}
where $c_i$ is determined by the normalization condition $v_i^Tv'_i = 0$. This expression can blow up if the matrix becomes defective, meaning your eigenvectors might change suddenly. This is why $A$ being normal would be very helpful--normal matrices are never defective.
