# Why is the eigenvalue equation $Ax = \lambda x$ nonlinear?

I read in a book that the eigenvalue equation is nonlinear but I can't see why that is form just looking at it...

$$Ax = \lambda x$$

looks a lot similar to

$$Ax = b$$

and if the matrix $$A$$ represents a linear transformation of $$x$$ then I don't see why $$Ax = \lambda x$$ would be nonlinear...

• for non-linear, may be referring to the characteristic polynomial equation, of which the eigenvalue $\lambda$ is a root (and that equation could be quadratic or cubic if the matrix $A$ is $2\times2$ or $3\times3$, for example) – J. W. Tanner Jul 31 at 1:31
• They probably mean that since $\lambda$ is unknown, $\lambda x$ is non-linear in terms of the unknowns $\lambda, x$. – Michael Biro Jul 31 at 1:32
• The characteristic polynomial equation for finding the eigenvales is nonlinear. While $Ax=\lambda x$ is a system of linear equations like you mention. – JBL Jul 31 at 1:35

It sounds like you're interested in why "equation problems" $$Ax=b$$ are called linear while "eigenvalue problems" $$Ax=\lambda\cdot x$$ are called nonlinear. There are likely many reasons to use this language, but here's one that invokes the notion of a linear combinations.

Suppose that $$x_1$$ and $$x_2$$ are two solutions to "equation problems" $$Ax_1=b_1$$ and $$Ax_2=b_2$$. Consider an arbitrary linear combination $$x$$ of $$x_1$$ and $$x_2$$, so $$x=c_1\cdot x_1+c_2\cdot x_2$$. If we define $$b=c_1\cdot b_1+c_2\cdot b_2$$, then $$Ax = A(c_1\cdot x_1+c_2\cdot x_2) = c_1\cdot Ax_1+c_2\cdot Ax_2 = c_1\cdot b_1+c_2\cdot b_2 = b$$ This illustrates that "linear combinations of solutions to equation problems are solutions to equation problems."

Now, suppose that $$x_1$$ and $$x_2$$ are two solutions to "eigenvalue problems" $$Ax_1=\lambda_1\cdot x_1$$ and $$Ax_2=\lambda_2\cdot x_2$$. Again consider an arbitrary linear combination $$x=c_1\cdot x_1+c_2\cdot x_2$$. This linear combination $$x$$ solves an eigenvalue problem if $$Ax=\lambda\cdot x$$ for some $$\lambda$$. However, we have $$Ax = A(c_1\cdot x_1+c_2\cdot x_2) = c_1\cdot Ax_1+c_2\cdot Ax_2 = c_1\cdot\lambda_1\cdot x_1+c_2\cdot\lambda_2\cdot x_2 \overset{?}{=} \lambda\cdot(c_1\cdot x_1+c_2\cdot x_2)=\lambda\cdot x$$ The $$?$$ in this equation indicates that a suitable $$\lambda$$ might not exist. Indeed, it isn't difficult to find exmamples where such a $$\lambda$$ does not exist. For instance, consider the data \begin{align*} A &= \left[\begin{array}{rr} 23 & 32 \\ -16 & -25 \end{array}\right] & \lambda_1 &= 7 & x_1 &= \left[\begin{array}{r} 2 \\ -1 \end{array}\right] & \lambda_2 &= -9 & x_2 &= \left[\begin{array}{r} 1 \\ -1 \end{array}\right] \end{align*} Note that $$Ax_1=\lambda_1\cdot x_1$$ and $$Ax_2=\lambda_2\cdot x_2$$. However, for $$x=x_1+x_2$$, we have $$\overset{A}{\left[\begin{array}{rr} 23 & 32 \\ -16 & -25 \end{array}\right]}\overset{x}{\left[\begin{array}{r} 3 \\ -2 \end{array}\right]} = \left[\begin{array}{r} 5 \\ 2 \end{array}\right] \neq \lambda\cdot x$$

Side Note. As mentioned in the comments, there are other reasons to call the eigenvalue problem $$Ax=\lambda\cdot x$$ nonlinear. For example, if one uses the characteristic polynomial $$\chi_A(t)=\det(t\cdot I_n-A)$$ to solve for the eigenvalues of $$A$$, then one ends up factoring an $$n$$th degree polynomial, which is a nonlinear problem.

It's also worth noting that taking any linear combination of two eigenvectors $$x_1$$ and $$x_2$$ corresponding to the same eigenvalue $$\lambda$$ does yield a solution to an eigenvalue problem. This is precisely the statement that eigenvectors corresponding to an eigenvalue $$\lambda$$ are organized into the eigenspace $$E_\lambda=\operatorname{Null}(\lambda\cdot I_n-A)$$.

The most remarkable thing about the eigenvalue equation $$AX=\lambda X ~~~~(1)$$ is that the eigen vector $$X$$ very specially comes back on the RHS with a multiplicative constant (a scalar called eigenvalue $$\lambda$$). On the other hand in an ordinary linear equation $$AX=Y ~~~~(2)$$ $$X$$ does not come back, it is some other vector (say) $$Y$$ which is never proprtional to $$X$$.

For the differential operator $$A=\frac{d}{dx}$$, $$f(x)= e^{ax}$$ is the only possible eigenfunction such that $$\frac{d}{dx}f(x)=a f(x),$$ where $$a$$ is the eigenvalue. Note the special status of the the exponential function here. Other functions $$\sin ax, \cos ax, \tan ax, e^{-ax^2}$$ cannot be the eigenfunction of this $$A$$, because they do not come back by differentiating them.

The operator $$A=\frac{d^2}{dx^2}$$ being second order has two linearly independent (LI) degenerate eigenfunctions as $$\sin ax$$ and $$\cos ax$$ with the same eigenvalue $$-a^2$$. Importantly any linear combination of these rwo $$X= A \sin ax + B \cos ax$$ is also an eigenfunction. This is why (1) is linear whereas $$AX=\lambda X^2$$ is nonlinear.

Also it has two more LI degenerate eigenfunctions $$e^{iax}$$ and $$e^{-iax}$$ with the same eigenvalue are eigenfunctions of $$\frac{d^2}{dx^2}$$ and so is any linear combination of them $$X=C e^{-iax} + D e^{iax}$$ is also its eigenfunction.

In case of several pairs of degenerate eigenfunctions any member of one pair is a linear combination of the members of other pair, for instance $$\cos ax= \frac{1}{2} e^{iax} +\frac{1}{2} e^{-iax}.$$

Suppose

$$A \in M_{n \times n}(\Bbb F), \; 0 \ne x \in \Bbb F^b, \; 1 \le n \in \Bbb N, \tag 0$$

for some field $$\Bbb F$$.

With

$$Ax = \lambda x, \; x \ne 0, \tag 1$$

we have

$$(A - \lambda I)x = Ax - \lambda x = 0, \tag 2$$

whence

$$x \in \ker (A - \lambda I) \Longrightarrow \det(A - \lambda I) = 0. \tag 3$$

Now

$$\det(A - xI) \in \Bbb F[x], \tag 4$$

and by virtue of (0), we have

$$\deg(A - xI) = n; \tag 5$$

thus for $$n > 1$$, $$\lambda$$ satisfies a polynomial of degree greater than one, hence the equation (3) for $$\lambda$$ is non-linear.

Of course, the mapping

$$A:\Bbb F^n \to \Bbb F^n, \; x \to Ax \tag 6$$

is linear in $$x$$; however, the mapping

$$\lambda x: \Bbb F \times \Bbb F^n \to \Bbb F^n, \; (\lambda, x) \to \lambda x \tag 7$$

is non-linear when seen as a function of both $$\lambda$$ and $$x$$, hence neither is the mapping

$$(A - \lambda I): \Bbb F \times \Bbb F^n \to \Bbb F^n, \; x \to (A - \lambda I)x; \tag 8$$

the process of forming $$\det(A - \lambda I)$$ transforms these higher-dimensional non linearities into a (typically non-linear) polynomial, which is often more tractable in terms of locating those $$\lambda \in \Bbb F$$ for which (2) binds.

Intuitively, the matrix equation is telling us that the matrix A acts on some vector x in a way that scales the vector linearly by some constant. That constant is called an eigenvalue. Eigenvalues are the zeroes of the characteristic polynomial. They are specific to the square matrix A.

A justification for the characteristic polynomial follows. The matrix equation is Ax=λx. Subtracting λx from both sides, we have Ax-λx=0 We multiply the λx by the identity matrix I with the same dimensions as A to get Ax-λIx=0 Factoring, we have (A-λI)x=0 Taking the determinant of both sides, we have det(A-λI)x=0. This equation generates the characteristic polynomial, which is nonlinear.

When you substitute the eigenvalues (λ) into the characteristic equation, you get 0, confirming that the eigenvalues are the zeroes of the characteristic polynomial, as stated.

The characteristic equation, which you solve to obtain $$\lambda$$, is always a polynomial, so that's what the book might have meant by nonlinear.

A linear system of equations, by definition, is one in which each term is a polynomial of total degree 1 in the unknowns. So it can contain linear combinations of terms of the form $$x_j$$, but not, for instance, $$x_j^2$$, $$x_ix_j$$, $$|x|$$, or $$e^x$$, ...

In the eigenvalue equation, you are looking for both $$\lambda$$ and the eigenvector $$x$$ at the same time. If you write out all terms explicitly, there are summands of the form $$\lambda x_i$$. Since $$\lambda$$ and $$x_i$$ are both unknowns, this is a product of two unknowns (exactly like $$x_ix_j$$ in the example above), and it is a nonlinear term.