# Can you modify a neural network in some way to produce a linear regression equation for the same input variables?

The full question is: Can you modify a shallow (1 hidden layer) fully connected feed-forward neural network in some way (either adding or removing some aspect of the neural network) to produce a linear regression equation for the same input variables?

I'm still working on getting information on what it means to modify and what the extent of those modifications can be. I'm also finding out if there are any restrictions on the number of inputs, hidden layer nodes, or outputs. I'll update this post soon when I get that information. Edit So any modifications to the neural net are okay such as weights, biases, number of hidden nodes, etc. as long as it still is a fully connected feedforward network with 1 hidden layer.

I know that the main difference between a neural network (NN) and a linear regression (LR) is that an NN has an activation function on each node which allows for non-linear functions to be applied so that the input data can better fit output data that is not linear (or classification data that can't be divided with a linear boundary). The set of multiple linear regression equations for multiple outputs looks like this:

The NN would look something like this with multiple inputs and outputs and only 1 hidden layer.

I'm thinking that a NN can achieve that same set of equations by forcing some of the weights and biases to be equal to 1 or 0 such that value that ultimately passes through to the final output layer is the same as the multiple linear regression equations. The activation function can simply be the linear activation function (which is akin to just not having one). The biases on all nodes in the hidden layer can be 0 while the biases on the final output layer could simply be retained to whatever their true value is.

Any thoughts on how to approach the problem? Thanks.

• Why would you want to do this (as opposed to just using linear regression directly, or just using neural networks)? May 30, 2022 at 2:15
• It was just a hypothetical conceptual question. In practicality, there are situations where the straightforward multiple linear regression is best and some where the non-linearity of the neural network is best. May 30, 2022 at 15:22