# Confusion about Matrix Diagonalisation, Generalised Eigenvalue problem and Quadratic Forms (Lagrangian example)

I'm apologize on behalf of the length of this question but I really wanted to pinpoint where the problem lies

## Part One (nonessential)

The origin of the problem comes form a problem in physics which is to diagonalise a Lagrangian (a special kind of quadratic form) of the form $$L=\frac{1}{2}\sum_{i,j}m_{ij}\dot{x}_i\dot{x}_j-\frac{1}{2}\sum_{i,j}k_{ij}x_ix_j=\frac{1}{2}\big(\dot{\vec{x}}|M\dot{\vec{x}}\big)-\frac{1}{2}\big(\vec{x}|K\vec{x}\big)$$

where $\vec{x}=\begin{pmatrix} x_1 \\ x_2 \\.\\.\\x_n \end{pmatrix}$ and $M=(m_{ij})$ and $K=(k_{ij})$, and where $M$ and $K$ is a positive definite symmetric matrices

the physicsy way of solving the problem would be to use the Euler-Lagrange equations to get $n$ differential equations of the second order, namely: $$M\ddot{\vec{x}}+K\vec{x}=0$$ in matrix form, since $M$ is positive definite, then it is invertible, so we write $$\ddot{\vec{x}}+M^{-1}K\vec{x}=0$$ now we wish to find some vector $\vec{a}$ that $$\ddot{\vec{a}}+M^{-1}K\vec{a}=\ddot{\vec{a}}+\omega_a^2\vec{a}=0$$ $\vec{a}$ is aka the eigenvector with it's eigenvalue $\omega_a^2$. For this particular case the differential equation is easy to solve because it becomes the equation of the harmonic oscilator so $$Q_1(t)=\vec{a}e^{i\omega_at}$$

this is effectively the process of diagonalising the matrix $M^-1K$ which we do via the standard Lin.Alg. method $$det(M^{-1}K-\lambda I)=0$$ since $M$ is positive definite $det(M)>0$ so by Binet-Cauchy $$det(M)det(M^{-1}K-\lambda I)=0$$ In this way we find (let's say the system is non degenerate) we find all those $\omega_1^2,\omega_2^2...\omega_n^2$ and $\vec{a_1},...\vec{a_n}$ and finally all the special solutions $Q_1(t),Q_2(t),...Q_3(t)$ for those eigenvectors (aka Modes) the ''vectors'' in our ''solution space'' (as some physicist would say) $$\{Q_1(t),...Q_n(t)\}$$ form a basis of our solution space, so any possible solution $x(t)$ can be represented as a linear combination of the vectors $Q_1(t)...Q_n(t)$ so we can write $(C_i \in \mathbb{C})$

$$x(t)=C_1Q_1(t)+C_2Q_2(t)+...C_nQ_n(t)$$

where $C_i$ are determined from the initial conditions. so far so good, there has been no change of basis.

## Part Two

the second way to approach this is to go back to our Lagrangian

$$L=\frac{1}{2}\sum_{i,j}m_{ij}\dot{x}_i\dot{x}_j-\frac{1}{2}\sum_{i,j}k_{ij}x_ix_j=\frac{1}{2}\big(\dot{\vec{x}}|M\dot{\vec{x}}\big)-\frac{1}{2}\big(\vec{x}|K\vec{x}\big)$$

and ask: can we change the basis in such a way that both matrices $M$ and $K$ become orthogonal? if we can$^{(*)}$, let's call those coordinates $q_i$ (what a coincidence) then we get a Lagrangian (quadratic form) of the type

$$L=\frac{1}{2}\sum_{i,j}m'_{ij}\delta_{ij}\dot{x_i}\dot{x_j}-\frac{1}{2}\sum_{i,j}k'_{ij}\delta_{ij}q_iq_j=\frac{1}{2}\big(\dot{\vec{q}}|D_M\dot{\vec{q}}\big)-\frac{1}{2}\big(\vec{q}|D_K\vec{q}\big)$$

This Langrangian gives us the equations of motion that are completely separated

$$\begin{pmatrix} m'_1 & & & \\ & ..& & \\& & & .. & \\ & & & & m'_n \end{pmatrix} \begin{pmatrix} \ddot{q_1(t)} \\ . \\.\\\ddot{q_n(t)} \end{pmatrix} + \begin{pmatrix} k'_1 & & & \\ & ..& & \\& & & .. & \\ & & & & k'_n \end{pmatrix} \begin{pmatrix} q_1(t) \\ . \\.\\q_n(t) \end{pmatrix}=0$$

which means that for every $q_i$ we get the solution $q_i(t)=Ce^{i\omega_it}$ ($\omega_i$ is $\sqrt(k'_i/m'_i)$ ofcourse)

so we basically got the same thing as in Part one. We write our solution $\vec{x}$ in the basis of vectors $q_i$ which is equivalent to representing it as $$x_i(t)=C_1a_{i1}q_1(t)+C_2a_{i2}q_2(t)+...C_na_{in}q_n(t)$$

We can write the whole of $\vec{x}$ as

$$\vec{x}=\begin{pmatrix} x_1 \\ x_2 \\.\\.\\x_n \end{pmatrix}=\begin{pmatrix} C_1a_{11}q_1(t)+C_2a_{12}q_2(t)+...C_na_{1n}q_n(t) \\ C_1a_{21}q_1(t)+C_2a_{22}q_2(t)+...C_na_{2n}q_n(t) \\.\\.\\C_1a_{n1}q_1(t)+C_2a_{n2}q_2(t)+...C_na_{nn}q_n(t) \end{pmatrix}=\begin{pmatrix} a_{11} &a_{12} & & \\a_{21}& a_{22}& & \\& & & .. & \\ & & & & a_{nn} \end{pmatrix}\begin{pmatrix} C_1q_1(t) \\ C_2q_2(t) \\.\\.\\C_nq_n(t) \end{pmatrix}=A\vec{Q}$$

now $C_1,...C_n$ are initial conditions in the system with basis $q_i$ (I think?)

so the matrix $A$ composed of our eigenvectors transforms $\{q_i\} \to \{x_i\}$ so our initial change of basis was actually $A^{-1}$

all quiet on the western front

## Part Three (problems start emerging)

in part two$^{(*)}$ we assumed such a change of basis can be done a then found the matrix $A^-1$ that was the transformation we were looking for. I will now try to prove that this kind of transformation can always be done (and get confused along the way..)

We can diagonalise $M$ with and orthogonal matrix $U$ because $M$ is real and symmetries, so:

$$M=S^TD_MS$$

now since $M$ is positive definite it has positive eigenvalues so there exist a matrix $S_{ij}=\frac{\delta_{ij}}{\sqrt{\mu_i}}$ (where $\mu_i$ are eigenvalues of the matrix $D_M$)

we can see that the matrix $US$ transform the matrix $M$ in to an unit matrix

$$(US)^TM(US)=I$$

The matrix $(US)^TK(US)$ is still symmetric because

$$k'_{ij}=\sum_{k,l}(US)_{ik}^Tk_{kl}(US)_{lj}=\sum_{k,l}(US)_{ki}k_{kl}(US)_{jl}^T=\sum_{k,l}(US)_{jl}^Tk_{lk}(US)_{ki}=k'_{ji}$$

and since $K'$ is symmetric there exist a orthogonal matrix $V$ such that $V^TK'V=(USV)^TK(USV)$

This proves that the matrix $(USV)$ diagonalises $M$ and $K$ simultaneously.

Finally for the questions!

1.) What are we actually doing with $S$? Is it a change of basis of some kind that shortens each basis vector by the factor $\frac{1}{\sqrt{\mu_i}}$ ? is it a transformation at all? Does this mean that we have effectively changed all the eigenvalues to 1!? (which make no sense) or does that mean that it is a vector of our basis who, when multiplied by it's eigenvalue gives 1?

is the matrix $A$ from the end of part two equal to the matrix $(USV)$?

Does part three mean that we can reduce every matrix (that is positive definite) to a identity matrix, if yes why? if no, why not? and what does it mean geometrically?

In part 3, are the eigenvectors of $M$ and $K$ the same vectors or are they different but are just diagonisable by the same transformation? Is this even possible?

In this answer there is bascically a connection between part two and part one, where @MichaelSeifert defines the vector $\vec{v}$ and $\vec{v'}$ and says the basis vectors may not necessarily be orthogonal (mean the basis of vectors that diagonalise the Lagrangian) $\vec{v}$ and $\vec{v'}$ are not the eigenvalues of the same matrix? What are they, what do they represent? Doesn't this clearly state that Normal modes are orthogonal? shouldn't this also follow from the Principal axis theorem? why are they always orthogonal? (proof reference?)

I hope my questions are meaningful and understandable, and again sorry for the length of the post, but I'm just super confused and can't get my head around this for a two weeks now...

Cheers

Note that there are two types of "diagonalization". One is for a linear transformation, the other is for a quadratic form.

If I didn't misunderstand, you're diagonalizing two quadratic forms simultaneously. The geometric meaning of this procedure is redefining measure, which is the inner product of basis vectors. We can usually denote it as $[(e_i,e_j)]$. It will completely determine the value of quadratic forms, for example we can write $$x^TAx = \sum_{i,j}x_ix_je_i^TAe_j$$ where $x_i$ is the ith entry of $x$. The usual inner product is $$x^Tx = \sum_{i,j}x_ix_je_i^Te_j$$, so we have defined a new inner product $<e_i, e_j> = e_i^TAe_j$.

If there exists invertible $P$ such that $P^TAP=B$, we say matrix A and B are congruent. Every positive definite matrix is congruent to $I$. And every positive definite matrix is symmetric so we have $T$ orthogonal such that $T^TAT=T^{-1}AT=D$, where $D$ is diagonal.

So to do the diagonalization of $M$ and $K$, we can first choose $P$ such that $P^TMP=I$. Then choose $T$ such that $T$ is orthogonal and $T^T(P^TKP)T = D$. Let $Q = PT$, $x = Qy$, then we have $$\dot{x}^TM\dot{x}+x^TKx=\dot{y}^T\dot{y}+y^TDy$$

In this context, $x=Qy$ is just a substitution that changes measure, and has nothing to do with eigenvectors(which comes from linear transformation context).

• Can you please elaborate on what you mean by saying that $x=Qy$ is a change of measure? do you mean redefining the scalar product by which we construct the quadratic form, and if so, how exactly is this a change of measure. Thanks in advance :) – Alexandar Solženjicin Mar 26 '18 at 15:51
• @AlexandarSolženjicin The measure is just $[e_i, e_j]$ which tells you what the inner product should be. – OrdinaryWitch Mar 26 '18 at 23:23
• so you are saying that by rescaling the vectors we are, in fact, re-definig the inner product? – Alexandar Solženjicin Mar 26 '18 at 23:47
• Yes. In one way you can say the measure matrix is changed. Or you can view it just as a substitution, which suffices to simplify calculation. – OrdinaryWitch Mar 26 '18 at 23:51
• I have edited it, is that clear now? – OrdinaryWitch Mar 27 '18 at 3:06