10
$\begingroup$

I've seen the term $x^T A x$ come up in a bunch of different areas of linear algebra, where A is a square and usually symmetric matrix. Places I've seen it include defining the Raleigh quotient, defining positive/negative semi-definite matrices, and in the derivation of PCA. I've also seen it sometimes referred to as describing a quadratic form.

Is there some general definition/ intuitive description of what $x^T A x$ means with respect to a vector and a matrix? My sort of vague understanding is that it describes how a vector is changed under a linear transformation defined by A (for example if A causes x to rotate 90 $^\circ$ then $x^T A x = 0$) but I can't seem to come up with a more precise or insightful description of $x^T A x$, and I'm surprised how little I could find online considering how often I see this term come up.

$\endgroup$
  • 3
    $\begingroup$ Do you know what a bilinear form is? $\endgroup$ – Brandon Thomas Van Over May 22 '18 at 22:01
  • $\begingroup$ No I don't. I will look into that thanks. $\endgroup$ – person1 May 22 '18 at 22:03
  • $\begingroup$ OK so I can see that you could describe $x^T A x$ as a bilinear map that takes a nxn matrix and a vector of size n to a real number but I'm still not sure how that illuminates the meaning of it. $\endgroup$ – person1 May 22 '18 at 22:08
  • $\begingroup$ In some sense, you can consider it the linear algebra analogue to group theory conjugation (that is, sending $h$ to $g^{-1}hg$ for $h, g$ elements of a group). $\endgroup$ – Bill Wallis May 22 '18 at 22:10
  • 4
    $\begingroup$ I know them as Real Quadratic Forms. $\endgroup$ – CyclotomicField May 22 '18 at 22:15
5
$\begingroup$

$$Ax$$ represents the vector $x$ after it has undergone the linear transform described by $A$ and

$$x^TAx$$ is the dot product of the original and the transformed vectors. This dot product is zero when the two vectors are orthogonal, and maximized/minimized when they are parallel/antiparallel.

If $x$ is an Eigenvector of $A$,

$$x^TAx=\lambda x^Tx=\lambda\|x\|^2.$$

More generally, $x$ is a linear combination of Eigenvectors $ue_1+ve_2+\cdots we_n$ and

$$x^TAx=\lambda_1u^2+\lambda_2v^2+\cdots\lambda_nw^2$$ is a canonical quadratic form. Depending on the signs of the lambdas, the equation $x^TAx=1$ describes an ellipsoid or some kind of hyperboloid.

$\endgroup$
  • $\begingroup$ In that last equation you wrote, are the $u^2$, $v^2$, $w^2$ the weights of each eigenvector in the linear combination, or do they represent the squared magnitudes of the eigenvectors? $\endgroup$ – person1 May 22 '18 at 22:57
  • $\begingroup$ If $A$ is real valued you need $A^T=A$ (A is symmetric) to give an orthonormal basis for $\mathbb{R}^n$ of eigenvectors for $A$. If $A$ is complex valued you need $A^*=A$ ( A is Hermetian) to give an orthonormal basis for $\mathbb{C}^n$ of eigenvectors of $A$. You're using $\{e_i\}$ are orthonormal in your expansion $$x^TAx=\lambda_1u^2+\lambda_2v^2+\cdots\lambda_nw^2$$ $\endgroup$ – N8tron May 23 '18 at 0:41
  • $\begingroup$ @person1: they are the wieights, i.e. the coordinates in the frame of the Eigenvectors. $\endgroup$ – Yves Daoust May 23 '18 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.