0
$\begingroup$

I ran an artificial neuron network on data with about 2,000 rows and 3 features. I got a R2 of .06 which is really low, but a good MSE of .41. Why are these performance evaluators of this model contradicting? ..Or what does this tell me about my model?

$\endgroup$

closed as off-topic by Javi, Alexander Gruber Apr 29 at 1:50

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – Alexander Gruber
If this question can be reworded to fit the rules in the help center, please edit the question.

1
$\begingroup$

On what basis do you believe a MSE of $.41$ is good? The MSE is the mean squared error. so this just tells you that the size of your residuals is on the order of $\sqrt{.41}.$ That's in whatever units your response variable is in, so it really has no meaning without additional context. (Is an average error of twenty dollars "good"? You need more information, right? Like it would be very good if the thing you were trying to predict had fluctuations of millions of dollars, but horrible if it was in pennies.)

So you need a scale to compare it to. One natural choice is to comparing it to the mean squared error you would get by just guessing the mean value every time. In other words, the variance of the data. This is essentially what R-squared is: it is the variance of the residuals divided by the variance of the data (subtracted from one so that bigger means better). The fact that you only get 0.06 here suggests that while your MSE is $.41,$ the variance of your data is itself only slightly larger than $.41,$ so that your model is not explaining much of the variation of the data.

Before using a neural net to attempt to tease out the nonlinear relationships, it would be good to try something simpler. When you run a linear regression what happens? Do you see any (linear) correlations between your response and features? Another thing is just plotting... did you plot your response against your features and see if there looked like there was any dependence there? If your features don't actually carry any information about the response, it doesn't matter how fancy of a model you use.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.