Where does $\Lambda=P^{-1}AP$ come from? How do we derive the fact that if a matrix is diagonalizable then we can diagonalize it with the formula $\Lambda = P^{-1}AP$, where $P$ is a block matrix whose columns are the eigenvectors of $A$?
I can see that if you start with $A$ and multiply by $P=\begin{bmatrix} v_1 & v_2 & \cdots & v_k\end{bmatrix}$, then you end up with $AP = \begin{bmatrix} \lambda_1v_1 & \lambda_2v_2 & \cdots & \lambda_kv_k\end{bmatrix}$.  I don't see why the next step should be to left multiply by $P^{-1}$.
Also, even if the formula $\Lambda = P^{-1}AP$ holds, how do we know this is the only why to produce a diagonal matrix from $A$?  Perhaps $A$ is also similar to a diagonal matrix whose entries are not the eigenvalues of $A$ (but still the product of whose diagonal entries is the determinant of $A$).  How can we prove that this is not possible -- or if it is possible, why is it not ever mentioned?
 A: Let $(e_1 \dots e_n)$ us the usual basis, and $X_k$ the $k$th column of the matrix $X$.
If $P^{-1}AP = D$ is diagonal, then taking the $k$th column of $P$ you get that:
$$
(P^{-1}AP)_k =
P^{-1}AP_k = D_k = \lambda_k e_k
\\ \implies AP_k = \lambda_k\times P e_k = \lambda_k P_k
$$
So the columns of $P$ are eigenvectors for $A$.
A: Here's the $2 \times 2$ case you can generalize:
Suppose $A$ is an arbitrary $2 \times 2$ matrix with two eigenvalues, $\lambda_1, \lambda_2$, and hence two corresponding eigenvectors, $v_1$ and $v_2$. Define a matrix $P$ whose columns are those two eigenvectors, 
\begin{equation} P =\left( \begin{matrix} v_1 & v_2 \\ | & | \end{matrix} \right) \end{equation}
where those vertical bars indicate the column.
Then the action of $A$ on $P$ is the same as the action of $A$ on the individual columns. As those columns are also eigenvectors we may write
\begin{equation} AP =\left( \begin{matrix} Av_1 & Av_2 \\ | & | \end{matrix} \right) = \left( \begin{matrix} \lambda_1v_1 & \lambda_2v_2 \\ | & | \end{matrix} \right) =  \underbrace{\left( \begin{matrix} v_1 & v_2 \\ | & | \end{matrix} \right)}_{P} \underbrace{\left( \begin{matrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{matrix} \right)}_{\Lambda} \end{equation}
where $\Lambda$ is the diagonal matrix of eigenvalues. That is, $AP = P\Lambda$. As $P$ is invertible (why?) we may write 
\begin{equation} A = P\Lambda P^{-1} \end{equation}
or
\begin{equation} \Lambda = P^{-1}A P \end{equation}

On the uniqueness of the diagonal matrix $\Lambda$, see Ian's comment above.
A: This has nothing to see with eigenvectors: it is the general change of basis formula for linear maps.
Let me give some details: suppose a linear map $f$ has matrix $A$ in some basis $\mathcal B$. Let $mathcal B'$ another basis; denote as $P$ the matrix with column vector $C_i$ equal to the coordinates (in basis $\mathcal B$) of the $i$-th vector of $\mathcal B'$. $P$ is called the change of basis matrix from $\mathcal B$ to $\mathcal B'$.
Now for any vector $x$ with column matrix $X$ in basis $\mathcal B$, $X'$ in $\mathcal B'$, one has:
$$X=PX'$$
Let $Y$ the column vector of $f(x)$ in basis $\mathcal B$. $Y$  and $X$  are related by $Y=AX$. Similarly if  $Y'$ is the column vector of $f(x)$ in basis $\mathcal B'$, one has $Y=PY'$.
Thus we can write:
$$Y=PY'=AX=A(PX'),\enspace\text{whence}\quad Y'=(P^{-1}AP)X$$
This proves the matrix of $f$  in basis $\mathcal B'$ is:
$$A'=P^{-1}AP.$$
