Matrix Representation of Inner Product In my readings I have on several occasions encountered references to a linear algebra theorem that runs as follows:
Let $g$ be a non-degenerate inner product on the real vector space $V$. Then, there exists a basis $e_1, \dots, e_n$ such that the matrix of $g$ is diagonal and whose diagonal entries are all either $-1$ or $1$
Despite having encountered this result several times, unfortunately, I have not had the luck of finding a precise reference to where this theorem is stated/proved. Can anyone provide a reference that proves this claim?
 A: This is a really well-known result, so I am a bit surprised you can't find it. Any decent book on (general) algebra should have it. For example Lang's Algebra, Ch. XV 'Structure of Bilinear Forms'. Or Knapp's Basic Algebra, Symmetric Bilinear Forms. Or Mac Lane and Birkhoff's Algebra, Chapter on Quadratic forms.
As André said, this is often cited as Sylvester's Law of Inertia, and the pair (or triple) of numbers of +1,-1 (and 0's) is called the signature. So the words sylvester law inertia signature should lead you to relevant search results.
\Edit: Let me add to Qiaochu's answer a proof of well-definedness of the signature. It seems this one is a bit more elegant than one finds in most books.
So we know of an orthogonal decomposition $V=V_+\oplus V_-$, with our nondegenerate form $B$ positive resp. negative definite on $V_+$ resp. $V_-$. Say they have dimension $p,q$ respectively. If $W$ is any subspace of $V$ on which $B$ is positive definite, then $B$ is both positive and negative definite on $W\cap V_-$, hence $W\cap V_-=0$. This implies $\dim W\leq \dim V-q=p$. Hence $p$ is the maximal dimension of subspaces on which $B$ is positive definite; so $p$ only depends on the form $B$. 
(If the form is degenerate, the number of zeros is just the kernel, and $p+q$ is the rank.)
A: You are probably referring to Sylvester's Law of Inertia.  Maybe search directly for that.  However, I would suggest first looking into the theorem that says a symmetric matrix is diagonalizable.  A proof is in almost all beginning Linear Algebra books, and you can find information on the Web in the usual places.
The whole thing is a generalization of the familiar "completing the square" procedure.
A: Here's a proof. Let $B : V \times V \to \mathbb{R}$ be a symmetric bilinear form. It suffices to show by induction that we can write $V = \text{span}(v) \oplus W$ for some nonzero vector $v$ such that $B(v, w) = 0$ for any $w \in W$ and such that $B(v, v) = 0, 1, -1$. 
Choose an inner product $\langle \cdot, \cdot \rangle$ on $V$ (unrelated to $B$) and let $S^{n-1}$ denote its unit sphere. Let $v \in S^{n-1}$ be such that $B(v, v)$ is maximal. Let $w$ be a unit vector orthogonal to $v$; then
$$B(v + tw, v + tw) = B(v, v) + 2t B(v, w) + t^2 B(w, w) \le (1 + t^2) B(v, v)$$
by hypothesis, hence $2 B(v, w) \le t(B(v, v) - B(w, w))$. Taking $t = 0$ gives $B(v, w) \le 0$, and applying this result to $-w$ gives $B(v, w) = 0$. Depending on the value of $B(v, v)$ we can rescale $v$ so that $B(v, v) = 0, 1, -1$ and take $W$ to be the orthogonal complement of $v$ (with respect to the inner product). 
Taking $B = \langle v, Aw \rangle$ where $A$ is symmetric and positive-definite, the intuitive content of the above proof is that $v$ is an axis of the ellipse $B(v, v) = 1$. Note that I haven't actually proven the full content of the law of inertia, which states in addition that the number of times $0, 1, -1$ appears doesn't depend on the choice of decomposition. 
A: A slight correction: inner products are always usually positive definite so there is always a basis relative to which the matrix representing the product is the identity matrix (diagonal with all $1$'s). What you are asking about is the fact that every non-degenerate symmetric bilinear form over the reals can be represented relative so some basis by a matrix that is the identity matrix.
This follows immediately from Gram-Schmidt orthogonalization. Since the bilinear form $(\cdot,\cdot)$ is symmetric, we can define two vectors to be orthogonal if $(x,y)=0$.
Then picking an arbitrary basis and running Gram-Schmidt orthogonalization gives you an orthonormal basis relative to that form, i.e. a basis $\{v_i\}$ such that $(v_i,v_j)=0$ if $i\neq j$ and $||(v_i,v_i)||=1$. Then since the matrix $M=(m_{ij})$ representing $(\cdot,\cdot)$ relative to a basis $\{v_i\}$ has the property that $m_{ij}=(v_i,v_j)$
(since $(v_i,v_j)=(v_i^TMv_j)$), it follows that $M$ is diagonal with non-zero entries of norm $1$. But over the reals, non-zero number of norm $1$ are $\{1,-1\}$.

This all generalizes to symmetric bilinear forms (degenerate ones have a bunch of $0$'s along the diagonal) over arbitrary fields, except that if you don't have a norm you can't normalize the basis so that all the vectors would have norm $1$.
Note that over the complex number, the inner product is NOT a symmetric bilinear form, but a conjugate-symmetric bilinear form; Gram-Schmidt orthogonalization has to be modified accordingly, but can still be made to work.
