The problems that arise over arbitrary fields mostly have nothing to do with linear algebra itself, more with the applications.
You have to realize that linear algebra arose as a conglomerate of many different concepts and applications: solving linear equations, linear transformations between vector spaces, general matrix theory, matrix groups and rings, geometric problems, engineering applications.
All of the algebra essentially only depends on the fact that you are working over a field. But when you are working with an arbitrary field, you often don't have a notion of distance, angles, slopes, etc. So
- anything that involves having something be greater than/less something else could create a problem (as mentioned in another answer, inner products can give problems because asking that $x \cdot x \geq 0$ for all $x$ can become meaningless)
- anything involving length requires a bit of consideration or redefinition (what does it mean to "normalize" a vector if you don't have a way to measure its length? How do you measure the distance between a vector and a subspace for a least-squares problem?)
Many things can be redefined however.
- Distance metrics such as the Hamming distance can be imposed (as well as others).
- Sometimes "normalizing" a vector can just mean scaling so the first (or last) nonzero entry is a 1.
- Orthogonality can be defined in terms of an arbitrary bilinear or sesquilinear form $B: V\times V \to \mathbb{F}$ (usually we require that $B$ is reflexive, that is that $B(x,y) = 0$ if and only if $B(y,x) = 0$ so that orthogonality is a symmetric relation).
Making these redefinitions will require verifying and reproving that you have analogous properties that you have in the real and complex case. Often you do have similarities, but often with some subtle differences (eg you can often have nonzero vectors that are orthogonal to themselves).