# Questions tagged [neural-networks]

For questions about the mathematics of artificial neural networks: their underlying multilayered graph object or their use as a data structure in machine learning algorithms. Consider also using the tags (machine-learning) or (graph-theory).

839 questions
Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

• 4,379
0 votes
2 answers
138 views

### Using Matrix Calculus in Backpropagation derivation. Rules of order of matmul and transposition when taking derivatives in different layouts.

I define a neural network with $L$ layers ($L-1$ hidden layers). The forward pass is as follows: $$\mathbf{a}^{(l)} = f(\mathbf{W}^{(l)}\mathbf{a}^{(l-1)}+\mathbf{b}^{(l)})$$ Where $l \in [0,L]$ and ...
• 45
0 votes
0 answers
18 views

### Gaussian Kernel outputs a 2*m feature map?

I am currently writing my masters thesis on the Double Descent Curve in Neural Networks and as I was doing some research, I came across the paper "On the Double Descent of Random Features Models ...
• 21
0 votes
0 answers
66 views

• 41
0 votes
0 answers
28 views

### closed loop operator for neural networks, final form.

I'm reading the book about neural networks from Simon Haykin. In the first section of recurrent neural networks plot this figure: and he says that the system is linear. So he discribes the system in ...
0 votes
0 answers
32 views

### What are RNN trying to do from a statistical point of view?

I know how recurrent neural networks work but let's say that I want to model their behaviour from a statistical point o view, how should I interpret their output? Surprisingly on the internet I found ...
1 vote
1 answer
95 views

### Neural ODEs-Dimension Mismatch in the proof of adjoint sensitivity method

In the paper 'Neural Ordinary Differential Equations' (2019) by Ricky T. Q. Chen et al., it is noted that adjoint variable $\mathbf{a}(t)$ as the gradient of the loss function with respect to the ...
• 19
0 votes
1 answer
53 views

0 votes
1 answer
51 views

### What am I misunderstanding in a paper using the Abel's functional equation?

I'm currently reading the paper available at https://arxiv.org/abs/1503.05724. In this paper, the authors define a function iteration as shown here: . They also discuss Abel's functional equation: . ...
• 282
2 votes
1 answer
87 views

• 83
-2 votes
1 answer
124 views

### A brilliant introductory course on machine learning (mathematical perspective) (simulation + implementation) [closed]

I am a grad student with a relatively good understanding of stochastic analysis / probability theory, but only basic coding experience. What is a good source (textbook or lecture notes) for an ...
• 444
0 votes
2 answers
110 views

### What are the formulas for ReLU, Tanh, and sigmoid activation functions [closed]

So I’m studying neural networks, and I came across these activation networks. Of course I could just ignore the mathematical specifics but they seemed interesting, especially since I like math. So ...
1 vote
0 answers
144 views

### Mathematical efforts to embed any object into n-dimensional Euclidean space (or spaces)?

There are efforts to automate proof discovery using deep neural networks. There are varied approaches how the mathematical objects can be embedded into the layer of neural networks. E.g. one can ...
• 1,313