physical significance of normalization in linear algebra

Question

When dealing with Eigen values and Eigen vectors, why is it preferred to normalize the Eigen vectors. What more/robust information does this operation provide?

What is its physical significance in simple/layman's terms. Does this also bring about a loss of information? Are there any constraints to using this operation in matrix and linear algebra?

Kindly educate.

Answer 1

A very well-known application in my area for eigenvectors is principal component analysis. We study the so-called co-variance matrix (which is symmetric) whose entries represent the relation between variables under study. The eigenvectors of this covariance matrix are called the principal components. In the application, they represent the principal directions which explain the observed data. This sense of "direction" is independent of the norm and is uniquely represented by the unit vector.

Answer 2

First of all, no, there is no loss of information when you normalize eigenvectors.

Firstly, an important example, the use of normalization in the case of a symmetrical matrix $A$ with the classical relationship:

$$A=PDP^{-1}$$

where you know that $P$ has for its columns, eigenvectors of $P$ which form a basis of $\mathbb{R}^n$, and, even more, an orthogonal basis of $\mathbb{R}^n$.

If we normalize these columns, $P$ becomes an orthogonal matrix, i.e., with property $P^TP=I \ \iff \ P^{-1}=P^T.$ Thus, relationship (1) becomes:

$$A=PDP^T$$

which is much simpler to use in certain computations (for example with quadratic forms, if Q:=P^T, the essential computation $X^TAX=X^TQ^TDQX=(QX)^TD(QX)=Y^TDY$ which allows the simplifications of the equations of conical curves, and their generalizations.)

Moreover, it is well known that orthogonal matrices are much more stable in numerical computations that other matrices because they have a condition number = 1 (https://en.wikipedia.org/wiki/Condition_number.)

A proof of the interest of people doing Numerical Analysis in (ortho)normalization is shown in different functions used in Matlab.

For example, let us have a look at the powerful "null" operator, giving a basis of the kernel of a matrix.

The script

M=[0 1 1 1

1 2 3 1];

null(M)

will "naturally" output a set of normalized (and even orthonormalized) vectors:

$$\begin{pmatrix}-0.21132&0.78868\\ -0.78868&-0.21132\\ 0.50000&-0.28868\\ 0.28868&0.50000\end{pmatrix} \ \ \text{though a more "simpler" one would be} \begin{pmatrix}1&1\\ 1&-1\\ -1&\ \ 0\\ 0&1\end{pmatrix}$$