Are there two different ways to generalize the orthogonal group?

Question: Is there more than one "appropriate" way to generalize the orthogonal group of $\mathbb{R}^n$?

The two competing ways which come to mind for me are:

1. The group of all isometries with the origin as a fixed point, for any (finite-dimensional) vector space $V$ with any metric $d$ (not necessarily inducing the vector space topology).

2. The group of all vector space automorphisms which preserve a given symmetric non-degenerate bilinear form (not necessarily positive-definite).

Are both of these generalizations in use? Do they have different names?

(Unnecessary) Background: There are at least two equivalent ways to define the orthogonal group for $\mathbb{R}^n$: the group of all isometries of $\mathbb{R}^n$ which have the origin as a fixed point (1)(Theorems 19 and 20, pp.137-138, Friedrich, Agricola, Elementary Geometry), or the group of vector space automorphisms of $\mathbb{R}^n$ which preserve the inner product (1)(2)(3).

That these are equivalent is a result of the special properties of the (standard) metric on $\mathbb{R}^n$: not only is it normable, but said norm even satisfies the parallelogram identity and thus is induced by an inner product.

What confuses me is that these two characterizations seem to provide very different directions of generalization.

We can always define a group of isometries for any metric space, irrespective of whether the metric is induced by a norm, much less of whether it is induced by a norm satisfying the parallelogram identity. Even the discrete metric has a well-defined isometry group. In practice it probably makes sense to restrict to those metrics inducing the vector space topology, however.

On the other hand, the other definition of orthogonal group generalizes to vector spaces equipped with any symmetric non-degenerate bilinear form (1)(2), even those which are not positive-definite, and thus don't induce a norm nor induce a metric (I think).

• The group of all isometries must act transitively on the unit sphere because it has a subgroup (the group of orthogonals) which acts transtively on the unit sphere. What the other group elements do is in some sense "forgotten" by the definition of transitive. About norms and inner products, are you aware of the parallelogram law condition? – Will R Mar 9 '17 at 15:00
• Of course, this is based on what Spivak means by transitive. As someone who just finished a group theory module, I would nitpick that the group of all isometries technically does not even act on the unit sphere at all, since we do not have a map $G\times S^{2}\to S^{2}$ (we are not guaranteed to end up in the codomain), and the same goes for the group of isometries fixing a particular point, unless that point happens to be the origin, in which case we arrive back at the orthogonal group. – Will R Mar 9 '17 at 15:09
• The wording of the first definition is a bit strange: The orthogonal group is the group of isometries of $\Bbb R^n$ that fix a particular point, not just the set of isometries that preserve at least one point. – Travis Mar 9 '17 at 15:09
• Ah, my apologies, on closer inspection I see that you are aware of the parallelogram condition. – Will R Mar 9 '17 at 15:18
• Yes, there is more than one generalization. This should not "confuse" you; it is the nature of things. For example, the group of matrices preserving the $\ell_1$-metric on $\mathbb{R}^n$ is the hyperoctahedral group consisting of all monomial matrices with entries in $\left\{1,-1,0\right\}$; this is not a group of matrices preserving a given bilinear form (I believe). – darij grinberg Mar 9 '17 at 19:17

For any $n$-by-$n$ matrix $M$, we can define an inner product on $\mathbb R^n$ given by $$\langle u,v\rangle_{\!_M} := u^{\top}Mv$$ Inner products define "distances" and "angles" by \begin{eqnarray*} \|u\|^2 &=& \langle u,u\rangle_{\!_M} \\ \\ \cos\theta &=& \frac{\langle u,v\rangle_{\!_M}}{\|u\|\, \|v\|} \end{eqnarray*} A linear transformation $T : \mathbb R^n \to \mathbb R^n$ preserves distances and angles if, and only if, $$\langle u,v\rangle_{\!_M} = \langle T u,T v\rangle_{\!_M}$$ for all $u,v \in \mathbb R^n$.

\begin{eqnarray*} \langle u,v\rangle_{\!_M} &=& \langle T u,T v\rangle_{\!_M} \\ \\ u^{\top}Mv &=& \left(Tu\right)^{\top}M\left(Tv\right) \\ \\ &=& u^{\top}\!\left(T^{\top} M T\right) v \end{eqnarray*} We need the set of all $T$ for which $M = T^{\top}MT$.

In the case of the standard Euclidean "dot product", $M=I$ and so we need $T^{\top}T=I$, i.e. the orthogonal matrices.

• This information is useful, I agree, but I do not see yet how it addresses the question. – Chill2Macht Mar 10 '17 at 12:20
• @William The point is that the two characterisations are the same. Isometries fix angles and distances. The inner product is a bilinear form is preserved by the orthogonal matrices. Angles and distances can be understood in terms of the inner product. 1 iff 2. – Fly by Night Mar 10 '17 at 16:19
• Yes but isn't that only true using the Euclidean metric (or a scalar multiple thereof)? – Chill2Macht Mar 10 '17 at 16:28
• @William It's true that in Euclidean space/geometry, length and angle are defined in terms of the Euclidean metric; that means $M$ is the identity in my answer. We can also talk about Minkowski space/geometry, where angles and distances have a different meaning to Euclidean geometry. The idea of angle and length comes from the Minkowski metric which has $$M = \left(\begin{array}{ccc} -1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{array}\right)$$ See Felix Klein's Erlangen Program en.wikipedia.org/wiki/Erlangen_program – Fly by Night Mar 10 '17 at 17:55
• Right, but I'm not talking about (pseudo)-Riemannian metrics, I'm talking about metric space metrics. The point is that in Euclidean space, the metric space structure alone (i.e. even before norm or inner product structure) is enough to define the orthogonal group. That this sparse definition (i.e. without any specific choice of bilinear form) leads to a group of isometries (metric space isometries) with the origin fixed with such nice properties is a result of the fact that the Euclidean metric (metric space metric) turns out to be normable for a norm satisfying the parallelogram identity. – Chill2Macht Mar 11 '17 at 8:39

EDIT: These questions (and their answers) seem strongly related. I could probably piece together an answer from them now if I were less lazy, but for now I will include the links for future reference:

When does the group of isometries of a norm determine the norm?

Isometry group of a norm is always contained in some Isometry group of an inner product?

Characterisation of inner products preserved by an automorphism

Orthogonal Matrices and Symplectic Matrices and Preservation of Forms

Of particular importance is the fact that the isometry group of the metric induced by an inner product determines the inner product up to a scalar multiple. /EDIT

This doesn't actually answer the question (hence the community wiki) but is a collection of some relevant thoughts I have had based on Spivak's Comprehensive Introduction to Differential Geometry Volume 2. Maybe it will help someone else to think of the answer. None of it is necessary to understand the question though, so I thought it might be prudent to separate from the question.

Motivation: Previously, I had been trying to identify a "geometric" criterion for a norm to be induced by an inner product, or for a metric to have a "sensible" notion of angle, and whether this was equivalent to being induced by an inner product. For the simple case of $\mathbb{R}^n$, where the only inner product up to scalings and rotations is the standard one, it seemed to me intuitively that, for norm-induced metrics, what allows one to distinguish those that are induced by inner products is the shape of their unit spheres. Namely, the unit sphere of any norm on $\mathbb{R}^n$ which is induced by an inner product is (I think) a sphere [update: according to Spivak, it is an ellipsoid, not necessarily a perfect sphere]. Spheres are also (the only?) shapes which are invariant under rotations and reflections, i.e. the orthogonal group. Obviously if we must define the orthogonal group in terms of an inner product then this is somewhat circular, but if the orthogonal group has a definition in terms of more elementary properties of the vector space which are independent of any choice of inner product then it might work.

What also came to mind is the definition of angle (in radians) from pre-calc: the ratio of the length of the arc which subtends the angle to the circumference of a subtending circle. In order for this to be well-defined, we need it to be invariant under rotations and reflections, which only occurs if the "generalized circle" (i.e. unit sphere) is invariant under rotations and reflections. Thus it seemed that perhaps a well-defined notion of angle could be reduced to a choice of metric or of norm (I'm not sure) whose unit spheres are invariant under the orthogonal group. Thus it became somewhat important to me to try to understand whether the orthogonal group could only be understood in terms of a choice of an inner product or only in terms of a certain subgroup of isometries.

I believe that this question may be asking the same thing: Is a faithful representation of the orthogonal group on a vector space equivalent to a choice of inner product? but as mentioned before I do not understand it well enough to say for certain.

I also think that the following theorem from p. 208, Volume 2 of Spivak's Comprehensive Introduction to Differential Geometry may be relevant:

Theorem. Let $F: V \to \mathbb{R}$ be a continuous Minkowski metric on an $n$-dimensional vector space $V$. Suppose that for all $p$ and $q$ in the unit sphere $\{v \in V: F(v)=1 \}$, there is a linear transformation $\phi: V \to V$ such that $\phi(p)=q$ and $F(\phi(v))=F(v)$ for all $v \in V$. Then $F$ is the norm determined by some positive definite inner product.

The definition of Minkowski metric used by Spivak does not appear to be standard (in particular, the term is usually reserved for the symmetric bilinear form of relativity). It describes something similar to a seminorm, in that it is a relaxation of a norm, but a Minkowski metric is required to separate points (a seminorm is not) but is not required to satisfy the triangle inequality (a seminorm) is. From p.200 of Volume 2 of Spivak:

A Minkowski metric on a vector space $V$ is a function $F: V \to \mathbb{R}$ such that $F(v)>0$ for all $v \not=0$ and $F(\lambda v)=|\lambda|F(v).$

Clearly $F$ is completely determined by a "unit sphere" $\{v: F(v)=1 \}$; the unit sphere is symmetric with respect to $0 \in V$ [i.e. invariant under reflections because of the absolute homogeneity of $F$], and intersects every ray through $0$ exactly once. Moreover, any such set is clearly the unit sphere for some $F$.

Spivak also goes on to explain that the absolute homogeneity of such a function $F$ allows one to define a notion of length for curves in $V$ which is independent of parametrization.

On p. 203 Spivak shows how the convexity of the unit ball of such an $F$ is equivalent to $F$ satisfying the triangle inequality, and thus being a norm. On pp. 204-205 he goes into the geometric significance of this further:

A Minkowski metric $F$ can be used to define a "distance" function on $V \times V$, by $(v,w) \mapsto F(v,w)$ (however, it is easy to see that this distance function satisfies the triangle inequality, and is consequently a metric, if and only if $F$ is a Banach space norm on $V$). This is just the procedure by which, in analytic geometry, we define the distance between two points in $\mathbb{R}^n$; in this case, we choose $F(x)=(\Sigma(x^i)^2)^{1/2}$, motivated, of course, by the Pythagorean theorem. After the Pythagorean theorem has been incorporated into our definition of distance in this way, it is interesting to ask what content, if any, remains to this theorem. The answer is, that the Pythagorean theorem has been declared true only for right triangles with sides parallel to the axes, but remains true for all right triangles. This is because, when $F(x)=(\Sigma(x^i)^2)^{1/2}$, the isometries of $\mathbb{R}^n$ are transitive on the unit sphere. That is to say, if $p$ and $q$ are in the unit sphere, then there is a linear transformation $\phi: \mathbb{R}^n \to \mathbb{R}^n$ with $\phi(p)=q$ and $F(\phi(x))=F(x)$ for all $x$.

This same transitivity property holds for any $F$ which is the norm $|| \cdot ||$ associated to a positive-definite inner product $\langle , \rangle$ on an $n$-dimensional vector space $V$, since there is an isomorphism $f: \mathbb{R}^n \to V$ such that $f^* \langle , \rangle$ is the usual inner product on $\mathbb{R}^n$ (Theorem I.9-3). It turns out that this property actually characterizes the Minkowski metrics $F$ which arise from inner products.