Question about basis transform and its direction This question has been bothering me for a while. It's in connection with diagonalisation: I can never remember, whether it's $A = PDP^{-1}$ or $A = P^{-1}AP$.
I have an explanation but it's wrong because it leads to the wrong formula but I don't see the flaw.
Can anyone please point out the mistake in the following argument?
Let $A$ be some finite-dimensional linear map given as a matrix. Say, it is, as usually is the case, given with respect to the standard basis. Let's call this basis Basis 1.
Let's assume it has as many different eigenvalues as its dimension $n$ so we can find $n$ distinct and linearly independent eigen vectors. They form a basis for the eigen space. Let's call this basis Basis 2.
So far so good. Now, $P$ is the matrix containing these eigen vectors.
Also, $P$ is a matrix that represents a change of basis:
If we apply $P$ to the first standard basis vector $(1,0,0,\dots, 0)^T$ then we get the first column of $P$ which is the first eigen vector. So we have
$$ P(e_i) = v_i$$
where $e_i$ are the basis vectors of Basis 1 and $v_i$ are the basis vectors of Basis 2. Therefore, $P$ transforms Basis 1 into Basis 2.
Now to get the same linear map $A$ but with respect to Basis 2 I want to

*

*take a vector with respect to Basis 1


*transform it into Basis 2


*apply $D$ (i.e. the map $A$ but expressed w.r.t. Basis 2)


*transform the result from Basis 2 back into Basis 1
Therefore, expressing steps 1)-4), we should have
$$ A = P^{-1}D P$$
But this is wrong: the correct formula is the other way around.
So where is my mistake?
 A: The error is in the conclusion

Therefore, $P$ transforms Basis 1 into Basis 2.

You have this backwards. Let's call the vectors in Basis 2 (which are eigenvectors of $A$), $v_1, \ldots, v_n$. In this scenario, the vector $e_i$ should be interpreted as a coordinate column vector with respect to Basis 2, not 1. This coordinate vector corresponds to the vector:
$$0v_1 + 0v_2 + \ldots + 0v_{i-1} + 1v_i + 0v_{i+1} + \ldots + 0v_n = v_i.$$
Multiplying by $P$ turns this into $v_i$, this same vector, only now it's in terms of the standard basis (Basis 1). That's how you can read it and recognise it as one of the eigenvectors you computed for $A$. So, $P$ changes coordinate vectors with respect to Basis 2 into coordinate vectors with respect to Basis 1.
Here's another way you can think of it: a matrix is diagonal if and only if the standard basis is a basis of eigenvectors of the matrix. Given $Pe_i$ is the $i$th eigenvector of $A$ in our basis, we expect $APe_i = \lambda_i Pe_i$, or in other words, $(P^{-1} AP)e_i = \lambda e_i$. That is, the standard basis is a basis of eigenvectors of $P^{-1}AP$, which makes it diagonal. Thus, the formula should be $D = P^{-1} A P$, or equivalently, $A = PDP^{-1}$.
A: Where your thinking made a fundamentally wrong turn is where you say "Therefore, $P$ transforms Basis 1 into Basis 2". While linear maps can act on (abstract) vectors, the change of basis matrix $P$ is just a matrix, and you should not think of it as transforming vectors. What matrices do is transforming elements of $K^n$ (where $K$ is your field, for instance $\Bbb R$ or $\Bbb C$), by left-multiplication into other such elements (possibly with a different $n$, though not here since $P$ is a square). Those elements of $K^n$ are most often, and certainly in the case of change-of-basis matrices, the (columns of) coordinates of vectors, which correspond to vectors once a basis is chosen.
Now then columns of $P$ are contain the coordinates of the vectors of basis 2 with respect to basis 1. The coordinates of any basis with respect to that basis itself are given by the columns of the identity matrix$~I_n$. Now if one left-multiplies any column of$~I_n$ by $P$ then the result is the corresponding column of $P$, and while it is therefore true to say that left-multiplying by $P$ the coordinates of the vectors of basis 1 with respect to basis 1 by $P$ results in the coordinates of the corresponding vectors of basis 2 with respect to basis 1, that is not useful thing to say. Instead you should think of it as follows: left-multiplying by $P$ the coordinates of a vector$~v$ of basis 2 with respect to basis 2 by $P$ results in the coordinates of $v$ with respect to basis 1. The nice thing about the latter statement is that (by linearity) it extends to all vectors $v$. So multiplying by $P$ transforms ordinates from (those with respect to) basis 2 to (those with respect to) basis 1, and not the other way around as you falsely go on to assume. Changing this around will put you on the right track. (I am old enough to remember the PDP-11 minicomputer from the 1970s, this serves me as a mnemonic for the right formula.)
It is still somewhat unsettling that multiplication by the change-of-basis matrix$~P$ from an old to a new basis converts coordinates with respect to the new basis into those (of the same vector) with respect to the old basis, which seems kind of backwards; for the "forwards" coordinate change you would need to multiply by$~P^{-1}$. You can use the following image to get some intuition for this. When measuring real-world quantities one can use numbers if the unit of measure is fixed and understood in the context; for instance speed limits are given as numbers, and (where I live) a limit of 30 is to be interpreted as 30 km/hour (claiming in your defence that you interpret as 30 m/s in S.I. units will not be accepted). The unit of measure is the (single element) basis vector, and the number is the coordinate of the quantity; you can somewhat forget there even are units as long as they don't change. But if you need to change from one unit to another, say one that is $p$ times as large, then the numbers for fixed quantities change (this is coordinate change due to change-of-basis), but the should be divided, not multiplied, by $p$ to make the conversion; multiplication by $p$ only serve to convert numbers from using new units into those using the old ones.
