In the analysis of the stability of a dynamical system I came across the following Jacobi matrix $J$ (the eigenvalues of which determine stability): \begin{equation} J=KP \end{equation} where $K$ is an $m\times m$ real diagonal matrix and \begin{equation} P=A\left(A^{T}A\right)^{-1}A^{T} \end{equation} is an orthogonal projection. The non-square matrix $A$ is an $m\times n$ real matrix, $m\geq n$.

The eigenvalues of $P$ are 1 with algebraic multiplicity $n$ and 0 with algebraic multiplicity $m-n$ (see this proof exploiting the idempotence of $P$ and this proof arguging the multiplicity).

However, when you consider the whole matrix $J=KP$, I am not sure about its eigenvalues.

I am particularly interested if anything can be said about the sign of the the largest nonzero eigenvalue of $J$, as this is what matters for stability analysis.


Let $\lambda_{\max}(A)$ be the largest singular value of $A$; when $A$ is symmetric this is the largest absolute value of the eigenvalues of $A$.

Since $P$ is a projection, $\lambda_{max}(J)\le \lambda_{\max}(K)$. Equality can be attained: take $P$ to be the identity, or assume that the range of $P$ contains the largest eigenvector of $K$.

A similar inequality holds for the the maximal eigenvalue $\sigma_{\max}$, provided that $\sigma_{\max}(K)\ge0$. To see this, notice that any eigenvalue of $J=KP$ is also an eigenvalue of the symmetric matrix $PKP$. Since $P$ is a projection, $\|Pu\|\le\|u\|$ and therefore $$ \sigma_{\max}(J) \le\sigma_{\max}(PKP) =\sup_{\|u\|\le1}\langle PKPu,u\rangle =\sup_{\|u\|\le1}\langle KPu,Pu\rangle \le\sup_{\|v\|\le1}\langle Kv,v\rangle =\sigma_{\max}(K). $$ For the eigenvalue statement between $KP$ and $PKP$, notice that if $KPv=\lambda v$, then $P^{1/2}KP^{1/2}(P^{1/2}v)=\lambda (P^{1/2}v)$. So all the eigenvalues of $KP$ are also eigenvalues of $P^{1/2}KP^{1/2}=PKP$, because $P$ is a projection. $A^{1/2}$ denotes the matrix square root of a nonnegative definite matrix $A$ (and $P$ is such, as a projection).

You can show that any \emph{nonzero} eigenvalue of $PKP$ is also an eigenvalue of $J$. It could be that $J$ is not diagonalisable, but this will happen only if there is a nonzero $v$ such that $Pv=v$ and $PKv=0$ yet $Kv\ne0$ ($v$ is the range of $P$ but $Kv$ is in the orthogonal complement of the range of $P$).

We will have equality in the second inequality if and only if the maximising $v$ (or \emph{a} such maximiser if there are many) is in the range of $P$. How far off we are depends on how close is the range of $P$ to a maximiser, and also on how close are the other eigenvalues of $K$ to the maximal one.

If $\sigma_{\max}(K)<0$, then taking $P=0$ shows that $\sigma_{\max}(J)$ can be larger than $\sigma_{\max}(K)$. But the same idea shows that $\sigma_{\min}(J)\ge \sigma_{\min}(K)$ if the latter is not positive.


$K$ negative definite ---> $\sigma_{\max}(J)\le0$, and it will be zero unless $P$ is the identity.

$K$ not negative definite ---> $0\le\sigma_{\max}(J) \le \sigma_{\max}(K)$. Note that it doesn't mean that $K$ is nonnegative definite, all it means is that it has at least one nonnegative eigenvalue.

There analogous inequalities for $\sigma_{\min}$ also hold.

  • $\begingroup$ Thank you, $\lambda_{max}\left(J\right)\leq\lambda_{max}\left(K\right)$ is a nice idea. However, this is only a statement about the absolute values of eigenvalues. This is not so helpful for my problem, as I care about the largest nonzero eigenvalue. A bound on that would be incredibly helpful. Unfortunately I have to live with the fact that $J$ is not symmetric, as it is dictated by the linear stability analysis of a dynamical system near an equilibrium point of interest. $\endgroup$ – Alexander Erlich Feb 14 '18 at 20:48
  • $\begingroup$ Can I also ask for a reference on the inequality? If you have a good book recommendation that would help also. $\endgroup$ – Alexander Erlich Feb 15 '18 at 9:24
  • $\begingroup$ Sorry, I don't really know books about this, but I am sure that more or less any linear algebra would have. You can have a look here en.wikipedia.org/wiki/Matrix_norm, the $\lambda_{\max}$ is the norm they call $\|A\|_{2,2}$, and they say it's sub-multiplicative: $\|KP\|\le \|K\|\|P\|$ and $\|P\|=1$ (or 0 if $P=0$). $\endgroup$ – YZS Feb 16 '18 at 12:00
  • $\begingroup$ I had a mistake in my answer. $J$ is not symmetric but in many cases it will have eigenvectors, I will modify the answer. $\endgroup$ – YZS Feb 16 '18 at 12:24
  • $\begingroup$ I was hoping you could please have a look at my comment/answer below. $\endgroup$ – Alexander Erlich Feb 19 '18 at 2:07

This is an extended comment on @YZS answer.

I appreciate that the largest singular values $\lambda_\text{max}$ satisfy $\lambda_{\text{max}}\left(KP\right)\leq\lambda_{\text{max}}\left(K\right)$. Let's say that we call \begin{equation} \sigma_{\text{max}} \end{equation} the largest non-zero eigenvalue. From running numerical tests, I strongly suspect that the eigenvalues are bounded in a very similar way to the singular values, perhaps even the same way, $\sigma_{\text{max}}\left(KP\right)\leq\sigma_{\text{max}}\left(K\right)$.

Consider this simulation result, which I found to be representative:

enter image description here

Here $\lambda_\text{max}$ are largest singular values and $\sigma_\text{max}$ largest non-zero eigenvalues. Every dot represents a randomly generated matrix $A$ of dimensions $6\times 10$, the entries of which were integers between $-100$ and $100$. For all simulations, $K$ was always the same diagonal matrix \begin{equation} K=\text{diag}(-9,-9,-8,-8,-5,-4,-4,-1,1,3). \end{equation} So clearly, \begin{equation} \sigma_{\text{max}}\left(K\right)=3,\quad\lambda_{\text{max}}\left(K\right)=9. \end{equation} I have repeated such simulations for various $K$, as well as different dimensions of $A$ (starting with a square and then increasing column size, growing $K$ accordingly). I have found that the stated inequalities for both singular and non-zero eigenvalues were always true in my simulations.

Of course such tests are not an exhaustive search, but can anyone prove or provide a plausibility argument why $\sigma_{\text{max}}\left(KP\right)\leq\sigma_{\text{max}}\left(K\right)$ might be correct?

Also, given that $KP$ is not symmetric, is it somehow obvious why I never find any imaginary eigenvalues in numerical simulations?

As an aside, as you change the dimensions of $A$ from a square to increasingly elongated rectangle, it appears that the largest nonzero eigenvalues $\sigma_\text{max}(K P)$ move further and further away from the (supposed) bound $\sigma_\text{max}(K)$. This trend also holds for the max. singular values. In the plot below, $\sigma_\text{max}(K)=-2$ and the dimension of $A$ changes from $6\times 6$ to $6 \times 40$, as the eigenvalues move further and further below the (supposed) bound $\sigma_\text{max}(K)$:

enter image description here

  • $\begingroup$ Nice. You can show that the inequality holds for the largest eigenvalue as well. In fact $KP$ and $PKP$ have the same nonzero eigenvalues, and we have $\sigma_{\max}(K)=\sup_{\|v\|\le1} \langle Kv,v\rangle$. But $\sigma_{\max}(PKP)=\sup_{\|u\|\le1}\langle KPu,Pu\rangle$ is less than $\sigma_{\max}(K)$ because $\|Pu\|\le\|u\|$. I will modify my answer accordingly. $\endgroup$ – YZS Feb 20 '18 at 15:27
  • $\begingroup$ Are you interested in $K$ negative definite ($\sigma_{\max}(K)<0$)? In that case, unless $P$ is the identity, the largest eigenvalue of $J$ is zero. I guess the largest nonzero eigenvalue of $J$ will have to be smaller (more negative) than $\sigma_{\max}(K)$, but I haven't tried out the details. $\endgroup$ – YZS Feb 20 '18 at 16:17
  • $\begingroup$ Thank you for updating your reply! No, my $K$ are not necessarily negative definite, as in the case of the first numerical plot. $\endgroup$ – Alexander Erlich Feb 20 '18 at 16:35
  • $\begingroup$ So in general the nonzero eigenvalues of $KP$ and $PKP$ are the same. Usually $KP$ will have $m$ eigenvalues that are real. Trouble can only occur if $K$ sends a vector in the range of $P$ to somewhere in the orthogonal complement: $K=\begin{pmatrix}1 & 0 \\0 & -1\end{pmatrix}$ and $P$ matrix of all $1/2$, then $KP$ is not diagonalisable. But if you take $K$ and/or $P$ to be random, then this situation is unlikely to occur. $\endgroup$ – YZS Feb 21 '18 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.