How do I convert a quadratic form to a diagonal form? I don't understand how I should choose the transformations to convert a quadratic form to a diagonal form.
Ex: $x_1\cdot x_2 + x_1\cdot x_3 + x_2\cdot x_3$ 
 A: If you just need the final diagonal form, then finding the eigenvalues, as in Joanpemo’s answer, suffices. To find the transformation that produces this form, you’ll also have to find corresponding eigenvectors—solve the equations $(A-I)v=0$ and $(A+\frac12I)v=0$—and then produce an orthonormal basis from them.  
Since the matrix you’re working with is real symmetic and thus diagonalizable, you’ll be able to find two linearly independent eigenvectors corresponding to  $-\frac12$. The eigenspaces of the two eigenvalues are orthogonal, so you can use the Gram-Schmidt process to produce the necessary orthonormal basis. Alternatively, since we’re working in $\mathbb R^3$, you can take the cross product of the eigenvector that you found for the eigenvalue $1$ with either of the eigenvectors that you found for $-\frac12$ and then normalize these three vectors. Once you have this orthonormal basis, the matrix of the diagonalizing transformation has these basis vectors as its columns. From there, you can read off the transformation in coordinate form, if necessary.
A: Take the form's matrix and diagonalize it. It is always possible to do since the matrix will be, of course, symmetric:
$$A=\begin{pmatrix}0&\frac12&\frac12\\\frac12&0&\frac12\\\frac12&\frac12&0\end{pmatrix}\implies \det(xI-A)=\begin{vmatrix}x&-\frac12&-\frac12\\-\frac12&x&-\frac12\\-\frac12&-\frac12&x\end{vmatrix}=x^3-\frac14-\frac34x=$$
$$(x-1)\left(x^2+x+\frac14\right)=(x-1)\left(x+\frac12\right)^2\implies\text{the matrix's diagonalization is}$$
$$\begin{pmatrix}1&0&0\\0&-\frac12&0\\0&0&-\frac12\end{pmatrix}$$
so your form can be expressed with new variables as
$$x^2-\frac12y^2-\frac12z^2$$
A: Let $p = x_1x_2 + x_1x_3 + x_2x_3$
Let $x_2 = y + x_1 \Leftrightarrow y = x_2 - x_1$
$p = x_1 (y+x_1) + x_1x_3 + (y+x_1)x_3 \\ \ \ = (x_1 + 0.5y + x_3)^2 - (0.5y)^2 - {x_3}^2 \\ \ \ = (0.5x_1 + 0.5x_2 + x_3)^2 - (0.5x_1 - 0.5x_2)^2 - {x_3}^2$
