What really is diagonalization?

Question

If I have a square matrix $A$ representing a linear transformation $T:V\rightarrow V$ w.r.t the basis, $B=\{v_1,v_2..,v_n\}$ and $A$ is Hermitian. So we have

$Av_n=\lambda_{n}v_n$

where $\{v_1,v_2..,v_n\}$. are eigenvectors and {$\lambda_1,\lambda_2..,\lambda_n$} are corresponding eigenvalues.

So the matrix $A$ is said to be diagonalizable iff there exists a matrix $P$ such that $P^{-1}AP=D$ is a diagonal matrix.

where matrix $P=\Big[v_1 v_2 ...v_n\Big]$ and $D$ is a diagonal matrix with the eigenvalues as its diagonal entries.

Hope its correct.

What exactly is happening in the diagonalization ?

My Understanding:

Is it like let's say

we have this transformation matrix $A$ corresponding to our actual basis $B$, comprising of our eigenvectors. We are changing the corresponding linearly independent basis to a convenient basis (say {C}) and we are finding the transformation matrix D (representing $T:V\rightarrow V$) in our new, more convenient basis,

$C=\{e_{1}=(1,0,...,0),e_{2}=(0,1,...,0),....,e_{n}=(0,0,...,1)\}$

So we have $Ce_{n}=\lambda_{n} e_{n}$.

The eigenvectors are different for matrix $D$ but the eigenvalues are the same as $A$.

Here the purpose of $P$ is to change the basis from $B$ to $C$ and finding the corresponding transformation matrix in the new basis. I think $P$ is Unitary since $A$ is Hermitian

Answer 1

A linear operator is not hermitian "with respect to a basis".

When there is a basis of eigenvectors as you mention, $Av_k=\lambda_kv_k$, then $A$ is already diagonal. The point of diagonalization is to find such basis.

Answer 2

My interpretation is that diagonalization, when possible, simplifies the interpretation (and subsequent computations) of a matrix. In the eigenbasis, operations can be performed "coordinate-wise", or basis vector by basis vector, almost independently.

Suppose you have a vector $x^a = (x_1,x_2,\dots,x_n)$, and you already have computed $y^a = Ax^a$. If you modify only one component of $x^a$, for instance $x^b = (x_1,7\times x_2,\dots,x_n)$, you might be forced to recompute $y^b = Ax^b$ in a non-eigenbasis. In an eigenbasis, you only need the recompute the second coordinate: $y^b_2 = 7\times y^a_2$, the others remaining unchanged.

This is one of the reasons why one sometimes first diagonalizes operators (or nearly diagonalizes), to enable faster and more accurate subsequent computations. An iportant example is related to the Fourier transform. A linear and time invariant system can be represented by a convolution operator (see Fourier transform as diagonalization of convolution). A convolution is not fully easy to interpret. Yet, complex exponentials (cisoids, cis($x$): "cosine plus i sine") are eigenfunctions for such systems.

So, if you choose an appropriate basis of cisoids, the convolution operator is diagonalized. Consequently, after the discovery of fast implementations for the Fourier transform (such as the FFT), due to the simplicity of operations in the Fourier basis (mere products instead of convolutions), many signal analysis and image filtering operations have been implemented in the Fourier domain. Think about mp3, JPEG, Shazam, which are by-products of the FFT.

In short, diagonalization is at the heard of the digital revolution.