I have been reading into orthogonal groups and have some questions.

Recall the orthogonal group of matrices $O(n)=\{A\in \mathbb{R}^{n,n} : \det{A}\ne 0, A^T=A^{-1}\}$. This satisfies $\langle Ax,Ay\rangle=x^TA^TAy=x^Ty=\langle x,y\rangle$ for all $x,y\in \mathbb{R}^n$, where the inner product given is the dot product.

Suppose we have another inner product on $\mathbb{R}^n$, say $[-,-]$. One can similarly define the orthogonal group of this inner product, $O([-,-])=\{A\in \mathbb{R}^{n,n}: \det{A}\ne 0,\quad [Ax,Ay]=[x,y]\quad\forall x,y\in \mathbb{R}^n\}$.

1) From the calculation in paragraph 1, we have $O(n)\subset O(<-,->)$ i.e. the orthogonal matrices are a subset of the matrices invariant with respect to the dot product). Can we say that these two definitions are equivalent? From that same calculation in paragraph 1, I only see the requirement that $A^TA=I_n$ (i.e. $A^T$ is a left inverse). Is this the best we can conclude? (I am pretty sure not!)

2) More generally, can we relate the orthogonal matrices to the arbitrary inner product $[-,-]$? Are there any general conditions for which $O(n)\subset O([-,-])?$ (or even $O(n)=O([-,-])?$

Thoughts: It is well known that any inner product on $\mathbb{R}^n$ is of the form $[x,y]=x^TBy$, where $B$ is a symmetric positive definite matrix. Therefore, if $A\in O(n)$, then $[Ax,Ay]=x^TA^TBAy$, so I guess the requirement that $A\in O([-,-])$ is that $A^TBA=B$. That is, $A^{-1}BA=B$. From here I have no idea how to proceed further - this still does not feel very intuitive to me!

Edit: it seems $A^TBA=B$ characterises the matrices in $O([-,-])$ according to Orthogonal matrices only defined for standard inner product?.


  • $\begingroup$ I think you can use the Gram-Schmidt process to construct a basis which is orthonormal with respect to your alternative form $[-,-]$. So under this change-of-basis, $O([-,-])$ is isomorphic to the standard group $O(n)$. $\endgroup$ – Nick Oct 4 '19 at 0:16
  • 1
    $\begingroup$ For 1), note that $A^TA=I_n$ iff $AA^T=I_n$ so the two notions are equivalent. $\endgroup$ – Mr Martingale Oct 5 '19 at 14:57

The condition $A^TBA=B$ is equivalent to $(B^{-1/2}A^TB^{1/2})(B^{1/2}AB^{-1/2})=I$. Therefore $$ O([-,-])=\left\{B^{-1/2}QB^{1/2}:\ Q\in O(n)\right\}. $$ Now suppose $A\in O(n)\cap O([-,-])$. Then $AB=A(A^TBA)=(AA^T)BA=BA$. It follows that $O(n)\subseteq O([-,-])$ if and only if $B$ commutes with all real orthogonal matrices, but this occurs only when $B$ is a scalar matrix and in that case, $O(n)=O([-,-])$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.