Left and right eigenvectors perpendicular to each other

I just read in a textbook on numerical methods that you can always have that the right eigenvectors of a matrix can be taken as orthonormal to the left eigenvectors for a diagonalisable matrix. This seems suprising to me. I would have assumed that you need to have that the matrix is normal or hermitian, but is this also possible for all matrices?

-
Can you be more explicit? What exactly does the book say? – Brian Fitzpatrick May 17 '14 at 20:21
It says: If you have a matrix $A$ that is diagonalisable with right eigenvectors $\phi_1,...,\phi_n$, you can always find left eigenvectors $\psi_1,...,\psi_n$ such that $\phi_i \psi_j = \delta_{i,j}$ ( this is of course the real case) – user66906 May 17 '14 at 20:23
Is there any hypothesis of distinct eigenvalues? Ah, I guess with geometric multiplicity $>1$ we can choose the eigenvectors to make it true. – Ted Shifrin May 17 '14 at 20:34
no, there is nothing about distinct eigenvalues – user66906 May 17 '14 at 20:37

What is a standard exercise is that when $\lambda\ne\mu$, with $x$ a $\lambda$-eigenvector for $A$ and $y$ a $\mu$-eigenvector for $y$, then $x\cdot y = 0$ (compute $Ax\cdot y$ two ways). But it wasn't obvious to me what happened with the $\lambda$ eigenvectors for both.
Consider the diagonalization $P^{-1}AP=\Lambda$. Then, transposing this equation, we obtain $Q^{-1}A^\top Q=\Lambda$, where $Q=(P^{-1})^\top$. The columns of $P$ and $Q$ are eigenvectors of $A$ and $A^\top$, respectively. But let's look at $Q^\top P$: the $ij$-entry will compute the dot product of the $i^{\text{th}}$ eigenvector of $Q$ and the $j^{\text{th}}$ eigenvector of $P$. But $Q^\top P = P^{-1}P = I$, which is the desired result.