1
$\begingroup$

When I was dealing with optimization problem and deciding whether to minimize MSE or MAE, I had the following question: Is the solution to Mean Squared Error also the one to Mean Absolute Error? If yes, why would we choose one model over the other?

$\endgroup$
1
$\begingroup$

The answer is no. The MAE and MSE problems are not generally equal.

It generally depends on the type of your problem and what you exactly pursue. For example let the following optimization problems ($x,x_0\in\Bbb R^2$)$${P1: MAE\\\min ||x-x_0||_1\\s.t.\\ ||x||_2=1}$$$${P2: MSE\\\min ||x-x_0||_2\\s.t.\\ ||x||_2=1}$$Both the problems $P_1$ and $P_2$ stand for minimizing the distance between our parameter to a desired value in some sense. As an example, let $x_0=(1,2)$. Then the optimal solutions of the two optimization problems are $$x_1^*\approx (0.7071,0.7071)\\x_2^*\approx (0.4472,0.8944)$$which are not equal. Also a plot of the error functions is as follows

enter image description here

$\endgroup$
2
$\begingroup$

Consider $x_1,\ldots,x_n\in \mathbb R$.

The solution of the MSE problem is $\text{argmin}_x \sum_{k=1}^n (x_i-x)^2 = \{\frac 1n \sum_{k=1}^n x_i\}$ which you might know as the sample mean.

The solution of the MAE problem is $\text{argmin}_x \sum_{k=1}^n |x_i-x| = \{\text{medians of }(x_1,\ldots,x_n) \}$.

In general, the sample mean is not a median of $(x_1,\ldots,x_n)$, so the solutions of both problems are different.


Using this result with the distribution that puts mass $\frac 1n$ on each sample, the difference between the two solutions must be less than $\displaystyle \sqrt{\frac 1n\sum_{k=1}^n (x_k-\overline x)^2}$ where $\overline x = \frac 1n \sum_{k=1}^n x_k$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.