Is the matrix $A^4+A^3-3A^2-3A$ invertible? 
Let $A$ be a real $5 \times 5$ matrix satisfying $A^3-4A^2+5A-2I=O$. Is the matrix $A^4+A^3-3A^2-3A$ invertible?

Consider the polynomial $t^3-4t^2+5t-2=(t-1)^2(t-2)$. The minimal polynomial of $A$ divides $(t-1)^2(t-2)$. Let $p(t)$ be the minimal polynomial of $A$.
If $p(x)=t-1$ or $t-2$, then $A$ is a scalar multiple of the identity and $A^4+A^3-3A^2-3A=-4I \text{ or }6I$, so $A^4+A^3-3A^2-3A$ is invertible.
If $p(t)=(t-1)^2(t-2)$ or $(t-1)(t-2)$, then $A$ is similar to a upper triangular matrix $J$ with diagonal entries $1,2$. Then $A^4+A^3-3A^2-3A=P(J^4+J^3-3J^2-3J)P^{-1}$ and $\det(A^4+A^3-3A^2-3A)=\det(J^4+J^3-3J^2-3J)$. Since $J^4+J^3-3J^2-3J$ is upper triangular and the diagonal entries do not vanish, $\det(J^4+J^3-3J^2-3J) \ne 0$.
If $p(t)=(t-1)^2$, then by the same reasoning, $A^4+A^3-3A^2-3A$ is invertible.
Is there more efficient way to solve this kind of problems or I have to discuss any possible situations every time?
 A: Since $A^3-4A^2+5A-2=0$, $A$ must be invertible ($0$ is not an eigenvalue, else we get $-2v=0$.) Since $$A^4+A^3-3A^2-3A=A(A^3+A^2-3A-3),$$ it is enough to show that $A^3+A^2-3A-3$ is invertible. This follows from \begin{align}
A^3+A^2-3A-3&=(A+1)(A+\sqrt3)(A-\sqrt3)\end{align} Since $-1$, $\pm\sqrt3$ are not eigenvalues of $A$ (do not satisfy the polynomial in $A$), this polynomial in $A$ is invertible.
A: In general, the eigenvalues (with algebraic multiplicity) of $P(A)$ for any polynomial $P$ are the values $P(\lambda)$ for every eigenvalue $\lambda$ of $A$. In particular, the first condition gives you that all eigenvalues of $A^3-4A^2+5A-2I$ are $0$, so
$$(\lambda-1)^2(\lambda-2)=\lambda^3-4\lambda^2+5\lambda-2=0$$
for every eigenvalue $\lambda$ of $A$, so $A$ can only have the eigenvalues $1$ and $2$. As a result, every eigenvalue of $A^4+A^3-3A^2-3A$ will be
$$\lambda^4+\lambda^3-3\lambda^2-3\lambda$$
for $\lambda\in\{1,2\}$, and you can manually check each $\lambda$.
A: If $P$ is a polynomial, the matrix $P(A)$ is invertible if and only if $P$ is prime to the minimal polynomial $\mu_A$.
Proof. If $P$ is prime to $\mu_A$, then $UP+V\mu_A=1$ for some polynomials $U,V$. Now evaluate in $A$.
Conversely, if $D$ is a non constant common divisor, then $P=QD, \mu_A=RD$, so $\mu_A(A)=R(A)D(A)=0$. Now, since $D$ is not constant, the degree of $R$ is strictly less than the degree of $\mu_A$, and $R(A)$ is not $0$.Now $P(A)=Q(A)D(A)$, so $P(a)R(A)=0$ and $P(A)$ cannot be invertible  (otherwise $R(A)=0$)
In your case, performing Euclid algorithm for example, you get that your polynomials are coprime, so you have invertibility.
A: The polynomials
$$ p(x) = x^3 -4 x^2 + 5x -2 = (x-1)^2(x-2) $$
$$ q(x) = x^4 + x^3 - 3 x^2 -3x = x(x+1)(x^2 - 3)$$
are coprime.
From Bézout's identitity, it follows that there exists two polynomials $u(x)$ and $v(x)$ such that
$$ u(x) p(x) + v(x) q(x) = 1$$
As $p(A) = 0$, it follows that $q(A)$ is invertible.
A pair of Bézout coefficients $u(x)$ and $v(x)$ can be computed by the extended Euclidean algorithm.
A: The long division of $x^4+x^3-3x^2-3x$ by $x^3-4x^2+5x-2$ leaves a remainder $12x^2-26x+10$, so your matrix to be checked for invertibility is
$$
2(6A^2-13A+5I)
$$
Suppose $v\ne0$ is in the null space of $6A^2-13A+5I$, so $6A^2v-13Av+5v=0$.
We also know that $A^3v-4A^2v+5Av-2v=0$. Multiplying the former by $A$, the latter by $6$ and subtracting, we obtain $11A^2v-25Av+12v=0$ and an elimination from the two “degree two” relations yields
$$
7Av-17v=0
$$
which means that $A$ should have $17/7$ as an eigenvalue. However, this is not possible, because an eigenvalue $\lambda$ of $A$ must satisfy $\lambda^3-4\lambda^2+5\lambda-2=0$.

The above doesn't use the Jordan canonical form. If you know it, you can prove that the eigenvalues of $p(A)$ (for $p$ a polynomial) have the form $p(\lambda)$ where $\lambda$ is an eigenvalue of $A$. Since the eigenvalues of $A$ must satisfy $(\lambda-1)^2(\lambda-2)=0$, they can only be $1$ or $2$; however
$$
x^4+x^3-3x^2-3x=x(x^2-3)(x+1)
$$
and neither $1$ nor $2$ are roots of this polynomial.
