I recently read that it is good to choose a neural network with weights initialized as Gaussian random variables with mean 0 and standard deviation $\frac{1}{\sqrt{n}}$ where $n$ is the number of inputs to a neuron. Assuming we use the sigmoid activation function with Gradient Descent and Backpropagation, why is this true?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.