Qiaochu's comment is certainly the first thing to point out: since quadratic forms and bilinear forms are not interchangeable in characteristic $2$, you need to think about which you really are interested in, at some point. (Do we use $a\otimes a+Q(a)$ as relations or $a\otimes a + B(a,a) $?) Geometrically the characteristic $2$ case is exceptional, but other nonzero characterstics (and characteristic $0$) typically go through just fine.
But to continue, We are expecting in the end to take $n$ basis elements of a vector space, make $2^n$ ordered monomials out of these to make a basis for the algebra, and to have anticommutation relations between the original $n$ basis elements. Switching to characteristic two makes the anticommutation relation between the generators into a commutation relation, and that makes the whole algebra commutative. The algebra is still finite dimensional, and hence Artinian.
(Edited to a new argument to simplify things, I hope.)
Now in a commutative Artinian algebra, the Jacobson radical coincides with the nilradical, so we should ask if there are any nilpotent elements.
We can say a lot if $F=F_2$ and we picked the base elements to square to $1$. The $2^n$ basis elements for the algebra form an abelian group $G$, and that means this Clifford algebra is the same as the group algebra $F[G]$ where $F$ has characteristic $2$. Already we know the group algebra $F[G]$ must contain nilpotent elements, by Maschke's theorem.
Since the augmentation ideal is generated by $1+g$ for all $g\in G$, and it's easy to see that $(1+g)^2=0$ for all $g$, we see the augmentation ideal is nilpotent, and consequently it is the unique maximal ideal of the ring: $F_2[G]$ is local.
If base elements square to something other than $1$, then I think things get more complicated: you lose the group ring picture. You might be able to study it as the quotient of an appropriate group ring, though.
If any of the base elements square to zero, I think it can be reduced to the nondegenerate case if the kernel of the form you're using behaves like it does in the characteristic $\neq 2$ case. The resulting Clifford algebra should just be the algebra for the original space modulo the kernel of its form.
I guess I have some interesting homework for myself then (and you may care to try it out too.)
Determine whether we want to use a quadratic form for our algebra, or a bilinear form. Contemplate the differences...
Double check that the "algebra modulo radical is algebra of underlying space modulo radical" equation still holds true for algebras with degeneracy in whatever algebra we decide on (quadratic form or bilinear form).
Addendum:
In researching this question, I ran across a promising discussion in Clifford Algebras and their Applications in Mathematical Physics:Volume 1: Algebra and Physics. The page I've linked (350 and 351) begins a discussion of "the right" construction to choose, albeit for a more general case using forms that aren't symmetric or antisymmetric.