Why do we need statistical models e.g. a linear model or some large neural network? If there is some probability distribuiton, and we have some important random variables associated with it, why do we need large bloated models like decision trees, support vector machines, and neural networks? Wouldn't it suffice to take many random variables and estimate their distributions to keep the parameters of these random variables to store everything which is occurring probabilistically?


1 Answer 1


Linear regression is a random variable, though?

$$Y_i\sim \mathcal N(X_i\beta, \sigma^2)$$

So after we have estimated the coefficients, everything is stored internally as a random variable. But it is a random variable whose parameters may vary based on the input.

Similarly, a neural network is a complex sequence of deterministic functions, that boils down to a linear regression model in the end anyway.

$$\begin{split}Y|\textbf x, \theta &\sim \mathcal N(\textbf w^T\textbf z(\textbf x),\sigma^2)\\ \textbf z(\textbf x)&=g(\textbf V\textbf x)\end{split}$$

where $g$ is some function such as the sigmoid function, tanh, or more recently ReLU. The above is a neural network with an input layer, a single hidden layer, and an output layer. More complicated feedforward neural networks just involve putting additional hidden layers, but it all boils down to some functional transformation.

Statistics deals with random quantities, so it comes as no surprise that everything related to regression is a random variable.

On the other hand, there are parts of statistics that don't deal with random variables, like hierarchical clustering.

  • $\begingroup$ Ah so they actually all are random variables in the end. Thank you! $\endgroup$ 2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.