# Investigations about the trace form

Let us (again) consider the bilinear form $\beta(A,B)=\operatorname{Tr}(AB)$ for $A,B \in \mathbb{F}^{n,n}$ (quadratic matrices over a field $\mathbb{F}$). I am interested in finding the biggest subspace $U \subset \mathbb{F}^{n,n}$ such that for all $A \in U: \beta(A,A)=\operatorname{Tr}(A^2)=0$.

-
What do you mean by "closed under addition and multiplication"? – Joel Cohen Jun 28 '11 at 14:50
That you can add linear combinations of matrices in $U$ and for the resulting matrix $B$ the fact $\beta(B,B)=0$ is still valid. – Listing Jun 28 '11 at 14:52
So if I understand correctly, you are looking for a generating set? – Joel Cohen Jun 28 '11 at 14:56
Yes I am looking for a basis of $U$. But if its not possible to give one, just the dimension of the maximal $U$ would already be great, too. – Listing Jun 28 '11 at 14:59
$Tr(A+B)$ will be $0$, because trace is additive, but $Tr((A+B)^2)$ will not be. Try basically any example. I'm hesitating to write out an answer, because this sort of thing is making it hard for me to understand what you're looking for. – David Speyer Jun 28 '11 at 17:17

By coincidence, I learned very recently that the trace form on $M_n(k)$ is nondegenerate iff the characteristic of $k$ does not divide $n$. This was a mistake: it is in fact easy to see that the bilinear form $x,y \in M_n(k) \mapsto \operatorname{Trace}(xy)$ is nondegenerate for all $n \in \mathbb{Z}^+$ and all fields $k$. Indeed, if $x \neq 0$, then it has some $(i,j)$ entry nonzero, and then if you multiply on the right by the matrix $E_{ji}$ which has $(j,i)$ entry equal to $1$ and all other entries $0$, you get a matrix with nonzero trace. I was thinking of the definition of a strongly separable algebra, which is a separable $k$-algebra with nondegenerate trace form. But the trace form on an arbitrary finite dimensional algebra is the "unreduced trace", i.e, the trace of $x \bullet$ acting $k$-linearly on $A$. When $A$ is a matrix algebra (or more generally a central simple algebra), this unreduced trace is precisely $\sqrt{[A:k]}$ times the reduced trace, so when $A = M_n(k)$ it is $n$ times the usual matrix trace. Of course, when the characteristic of $k$ is divisible by $n$, this makes the unreduced trace identically zero, so $M_n(k)$ is not "strongly separable" (but the definition looks a little strange to me now, since it seems like we are focusing our attention on the wrong trace form).

I will assume throughout that the characteristic of $k$ is not $2$ so that the standard algebraic theory of quadratic forms can be applied.

You are asking for the maximal dimension of a totally isotropic subspace. If your quadratic form is nondegenerate, every totally isotropic subspace $U$ has an "isotropic supplement" $U'$ such that $U \cap U' = 0$, $\dim U = \dim U'$ and the quadratic form restricted to $U + U'$ is an orthogonal direct sum of $\dim U$ copies of the hyperbolic plane $\mathbb{H} = \langle 1, - 1 \rangle$. (See e.g. $\S 6$ of these notes on quadratic forms.) Therefore the dimension of a maximal totally isotropic subspace is equal to the number, say $r$, such that the Witt Decomposition of $q$ is

$q \cong \bigoplus_{i=1}^r \mathbb{H} \oplus q'$,

where $q'$ is anisotropic, i.e., $q'(x) = 0 \implies x = 0$.

So we want to know the Witt Decomposition of the trace form. When $k = \mathbb{R}$, joriki's answer to your previous question shows that

$q \cong \left(\frac{n^2+n}{2} \right) \langle 1 \rangle \oplus \left( \frac{n^2-n}{2} \right) \langle -1 \rangle \cong \left( \frac{n^2-n}{2} \right) \mathbb{H} + \left(n \right) \langle 1 \rangle$,

so the number $r$ is $\frac{n^2-n}{2}$. This is different from the formula you have given -- in fact, eventually smaller -- so if I have not made a mistake then you have: you should check first of all that the subspace you have in mind is really totally isotropic.

The next order of business is to compute the Witt Decomposition for the trace form over a more general field. Looking at the "matrix units" $E_{ij}$ as joriki did when $k = \mathbb{R}$ seems like a good start, but I haven't done this computation myself.

-
@Listing: no problem. Let me emphasize that such a maximal subspace is not unique: there will be infinitely many such subspaces, all conjugate under the orthogonal group of the quadratic form. – Pete L. Clark Jun 28 '11 at 17:46
For an example, I think the subspace of upper triangular matrices with $0$ on the diagonal fits the bill. – Joel Cohen Jun 28 '11 at 21:58
Yes I deleted the comments. Now everything is clear, thank you. – Listing Jun 30 '11 at 10:21