Neural network can be represented by the function of the following form:


where x is vector of input values, O - scalar output, M1...Mn are matrices and b1...bn are vectors and f1...fn are scalar (and usually nonlinear - sigmoid, hyperbolic tangent or step - functions) functions that are applied componentwise to the the vector (that arise from the matrix operations x*M1+b1 or fi(...)*Mi+bi). This function can be simpler for convolution neural networks where some elements of matrices are zeros and this function can be more involved for recursive neural networks.

My question is - are there research trends and research papers that studies exactly such functions? I am aware of the work https://www.cambridge.org/gb/academic/subjects/computer-science/pattern-recognition-and-machine-learning/neural-network-learning-theoretical-foundations?format=HB&isbn=9780521573535 but I would like to see more generalizations, more recent trends from the mathematical point of view that tries to generalize.

I am aware that spin glasses were used as models for neural networks (in quantum Hamiltonian formalism), but spins take only two values, but neurons may output values in real interval, so, spin glasses are very constrained models.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.