Is diagonalisation possible without an inner product? I have a gap in my education and I need help.
I am an engineer and I've been taught to diagonalise matrices by solving
$$
\textrm{det}(A-\lambda\cdot I) = 0
$$
for all possible $\lambda$'s, and hoping that each $\lambda$ gives you a unique eigenvector, and that this happens for symmetric matrices. I was lacking rigour, so I started to read books on linear algebra on my own but one thing that confuses me, is that they always prove diagonalisation with inner products, i.e. a matrix is diagonalisable iff it is self-adjoint (for base-field $\mathbb{R}$) or normal (for $\mathbb{C}$). But what if there is no inner product or the base-field is $\mathbb{F}_q$? Then I would rather use the "engineer"-method. Is there a way to make this rigorous? To use the symmetry of the matrix and the determinant, instead of inner products? Which books teach it that way?
 A: You can always find eigenvalues by 'the engineer method'. The rigorous 'mathematical methods' aren't really methods, they're just proofs that demonstrate that the set of eigenvectors spans the whole space (under certain conditions) - so you can properly represent your matrix / operator on the eigenbasis, which yields a lovely diagonal form.
So you will always be able to diagonalise a matrix on some subspace (the span of the eigenbasis) but it may not be the case that the whole matrix can be made diagonal (e.g, if the matrix is not symmetric). Such matrices are called defective, I think.
In $\Bbb C$, and probably any algebraically closed field, you can always have a Jordan normal form for your matrix - which is almost diagonal, but it's not even in true in $\Bbb C$-spaces that every matrix is diagonalisable. So you can't expect, necessarily, that you can always diagonalise your matrix over any field you like, because it fails on the 'nice' fields - nevermind the pathological ones.
The essential conditions for whether or not you can diagonalise an $n\times n$ matrix are:

*

*a) Does it have $n$ eigenvalues, up to multiplicity?

*b) Does each eigenvalue have the same geometric multiplicity as its algebraic multiplicity? If $\lambda$ is an eigenvalue with multiplicity $k$, are there $k$ linearly independent eigenvectors for $\lambda$?

The matrix is diagonalisable iff. $a,b$ hold, in any field. Side-note: you can pass to an algebraic closure to define algebraic multiplicity in general. The theorems proven in $\Bbb C$ with inner products just show that these conditions holds for any symmetric or unitary matrix. In the greatest generality however, it is $a,b$ that you must verify. For example, if the field isn't algebraically closed then it is entirely possible that $(a)$ fails.
Consider $\Bbb F_2$. The following matrix is symmetric: $$\begin{pmatrix}1&1\\1&1\end{pmatrix}$$But it is not diagonalisable. It has only $0,1$ as a possible eigenvalue - it turns out that "both" eigenvalues are zero. The corresponding eigenspace is $x=-y$, that is, the span of $(1,1)^T$ as $1=-1$. Clearly, this span is not the whole space $(\Bbb F_2)^2$, even though the matrix is symmetric. And, if it were diagonalisable, the diagonal matrix would have to be the zero matrix - this won't do. So a lack of inner product is a very real obstacle to the spectral theorem - we've just observed the spectral theorem fails in $\Bbb F_2$. I'd wager it fails in most, if not all, of the finite fields.
