There are many different models for neural dynamics. It is common to see a lot of stressing of neural networks, and there are certainly a lot of interesting questions you can ask about these mathematical structures (recurrent networks can be Turing complete, nonrecurrent networks can approximate a function to any given delta in a given range, given sufficient nodes, etc.).
But the brain operates on many different levels. At the cellular level, much of the operation is chemical reaction networks (metabolism), which are differential equations with very simple interpretations in terms of reactants and products, but when combined in large metabolic graphs can demonstrate many interesting phenomena. Neurons group into larger modular architectonic structures that fulfill larger functional computation roles.
At a higher level, dynamic epistemic logics and other temporal logics can be used to describe beliefs and belief revision in the face of sensory input. Rewriting logics have been used to great extent here. Building effective state machines and the automata of thought is still in it's infancy but shows a lot of promise in bringing the semantic layer into machine learning. These kinds of approaches also do not show as much reliance on the abstract statistical partitioning one sees in a lot of the pattern recognition literature, if that is anathema.
I'd recommend taking a look at Arbib, Erdi, and Szentagothai's seminal "Neural Organization: Structure, Function, and Dynamics" if you are interested in these approach to mathematical modelling of neural ontology.