to prove $f(P^{-1}AP)=P^{-1}f(A)P$ for an $n\times{n}$ square matrix? let $f(X)$ be a polynomial and let $A$ be $n\times n$ matrix.We have to show that for any $n\times n$ invertible matrix $P$,
$f(P^{-1}AP)=P^{-1}f(A)P$
and that there exist a unitary matrix $U$ such that both $U^*AU$ and $U^*f(A)U$ are upper triangular, where $U^*$ is conjugate transpose of $U$ and $P^{-1}$ is inverse of $P$ ..(m having a little idea about its prove..probably we'll use a result here that is "let A and B be n*n matrices s.that $AB=BA$ ..if all the eign values of A are distinct then $B$ can be expressed uniquely as a polynomial in $A$ with degree no more then $n-1$").
 A: Hint. Clearly,
$$
PA^2P^{-1}=PAP^{-1}PAP^{-1}=(PAP^{-1})^2.
$$
In general
$$
PA^nP^{-1}=(PAP^{-1})^n.
$$
Thus
$$
Pf(A)P^{-1}=P(c_nA^n+\cdots+c_1A+c_0I)P^{-1}=c_nPA^nP^{-1}+\cdots+c_1PAP^{-1}+c_0I=
c_n(PAP^{-1})^n+\cdots+c_1(PAP^{-1})^1+c_0I=f(PAP^{-1}).
$$
A: I show part of the proofs.
let $f(x)=\alpha_0 I +\alpha_1 X + \alpha_2 X^2 +...+ \alpha_n X^n$ 
Then $f(A)=\alpha_0 I +\alpha_1 A + \alpha_2 A^2 +...+ \alpha_n A^n$
Ok, let us see the term $A^n$, $A^n=(P^{-1}AP)^n=P^{-1}A^nP$.
So, we see that the polynomial $f(A)=\alpha_0 I+\alpha_1 P^{-1}AP +P^{-1}A^2P+...+P^{-1}A^nP=P^{-1} (\alpha_0 I +\alpha_1 A + \alpha_2 A^2 +...+ \alpha_n A^n)P = P^{-1}f(A)P$
A: Hint : Proceed by induction on $n$. For $n = 1$, there is nothing to do. Assume that the result holds up to an integer $n − 1, n \ge 2$, and prove that it also holds for $n$.
Given a family of commuting matrices $F$, such that $\forall A,B \in F$ ,$AB=BA$. Show that they possess a common eigenvector, which can be assumed to be normalized. Call this vector $v_1$ and form
an orthonormal basis $v = (v_1 , v_2 , . . . , v_n )$. Each $A \in F$, is unitarily equivalent to the matrix of
$x (\in \mathbb{C}^n) \rightarrow Ax \in \mathbb{C}^n$ relative to the basis $v$, i.e.
$\begin{pmatrix}a&x\\{0_{(n-1)\times 1}}&{A^*}_{(n-1)\times (n-1)}\\\end{pmatrix}$
By looking at the product $AB$ and $BA$ for all $A, B \in F$, we see that $AB = BA$ for all $A^*, B^* ∈ F^*$.
Thus, the family $F^* := \{A^*| A ∈ F \}$ is a commuting family of matrices of size $(n − 1)\times (n-1)$. By the
induction hypothesis, the family $F^*$ is simultaneously unitarily upper triangularizable. Infer that the family $F$ is itself simultaneously unitarily upper triangularizable (see the argument
in the proof of Schur’s theorem). This finishes the proof by induction.
