Neural network can be represented by the function of the following form:
x is vector of input values,
O - scalar output,
M1...Mn are matrices and
b1...bn are vectors and
f1...fn are scalar (and usually nonlinear - sigmoid, hyperbolic tangent or step - functions) functions that are applied componentwise to the the vector (that arise from the matrix operations
fi(...)*Mi+bi). This function can be simpler for convolution neural networks where some elements of matrices are zeros and this function can be more involved for recursive neural networks.
My question is - are there research trends and research papers that studies exactly such functions? I am aware of the work https://www.cambridge.org/gb/academic/subjects/computer-science/pattern-recognition-and-machine-learning/neural-network-learning-theoretical-foundations?format=HB&isbn=9780521573535 but I would like to see more generalizations, more recent trends from the mathematical point of view that tries to generalize.
I am aware that spin glasses were used as models for neural networks (in quantum Hamiltonian formalism), but spins take only two values, but neurons may output values in real interval, so, spin glasses are very constrained models.