# How to calculate relative error when true value is zero?

How do I calculate relative error when the true value is zero?

Say I have $x_{true} = 0$ and $x_{test}$. If I define relative error as:

$\text{relative error} = \frac{x_{true}-x_{test}}{x_{true}}$

Then the relative error is always undefined. If instead I use the definition:

$\text{relative error} = \frac{x_{true}-x_{test}}{x_{test}}$

Then the relative error is always 100%. Both methods seem useless. Is there another alternative?

• you need a maximum for that.. – Seyhmus Güngören Feb 15 '14 at 23:06
• Simple and interesting question, indeed. Could you tell in which context you face this situation ? Depending on your answer, there are possible alternatives. – Claude Leibovici Feb 16 '14 at 6:24
• @ClaudeLeibovici: I am doing a parameter estimation problem. I know the true parameter value ($x_{true}$), and I have simulation data from which I infer an estimate of the parameter ($x_{test}$). I want to quantify the error, and it seems that for my particular case relative error is more meaningful than absolute error. – okj Feb 17 '14 at 14:05
• What about $\text{error} = 2 \frac{x_{true}-x_{test}}{x_{true}+x_{test}}$ if it is for an a posteriori analysis ? – Claude Leibovici Feb 17 '14 at 14:16
• @okj. I am familiar with this situation. Either use the classical relative error and return $NaN$ if $x_{true}=0$ either adopt this small thing. It is always the same problem with that. You also can add a translation to the $x$'s to get rid of this. – Claude Leibovici Feb 17 '14 at 15:40

First of all, let me precise that I am not a statistician but a physicist very concerned by numerical issues in particular in the area of fitting data to models.

So, first consider that you have $[X(i),Y(i)]$ data points and that you want to adjust a model such as $$Y =a+b X+c X^2$$ Among your data points, you have one for which $Y(i)=0$. If you know that, for a specific and defined value of $X=x$, your model must return $Y=0$, you must include this condition and rewrite you model as $$Y=b (X-x)+c (X-x)^2$$ When doing the a posteriori analysis, you should not consider the data point $[x,0]$ since, by construction, it has been excluded from the data set by the constraint (you can even eliminate the data point from the data set; this will not change your results at all).

The other problem is more general. When your $Y(i)$ are almost of the same order of magnitude, the errors which define the objective function (say the sum of squares) is not very important. But, if the $Y(i)$ cover a very large range, minimizing the sum of squares of residuals give an incredible weight to the highest values and the small values of $Y$ play very little role; so typically, the low values are quite poorly represented by the model.

If you want all data points to be represented with the "same" quality of fit, weighted regression is required. By myself, what I use to do is to systematically minimize the sum of the squares of relative errors and, here, we come to your specific question : what to do if, for one data point, $Y=0$ ? I faced this situation in model for which no constraint was evident and I so decided, long long time ago, to define the relative error as $$\Delta =2 \frac{{Y_{cal}}-{Y_{exp}}}{{Y_{cal}}+{Y_{exp}}}$$ If the absolute error is small, this does not make any difference; if the absolute error is large, this bounds the error to human size.

I hope and wish these few notes will be of some help to you. Do not hesitate to post if you want to contiue this discussion.

• In my case, this shifts the problem to where Y_cal + Y_exp is near zero. (However, in my case I am trying to approximate a signal that is essentially a ripple on top of another signal, which happens to be non-negative everywhere, and I could normalise by the magnitude of that signal. So, the denominator in your last formula above is a kind of a weight, and I could use a different weight.) – Evgeni Sergeev Jun 4 '14 at 2:19
• Actually, if we use max(|Y_cal|, |Y_exp|) as the denominator, it might avoid the problems. – Evgeni Sergeev Jun 4 '14 at 4:02
• I'll take that back. That also doesn't help, because this bounds the error to be in the range [0,2], and wherever one of Y_cal, Y_exp is zero, the error normalised this way will be 1, in other words "very large". (Try plotting it for two lines, with slight relative offset to each other, crossing zero.) See my answer below for a solution. – Evgeni Sergeev Jun 18 '14 at 6:13
• This looks reasonable to me. However, I am working on a prediction problem for university project and I would be glad to know if there is some paper which explains why this should /could be used. I need to add references, more formal ones than an answer on question. Thank you in advance! – giliev Dec 22 '15 at 22:07

If this is based on any kind of real-world situation, then there should be multiple $x_{test}$ measurements, i.e. a distribution. Then it will have a standard deviation, or at least quantiles, and you can define the distance from the mean of the $x_{test}$ to $x_{true}$ in terms of these. E.g., $(\mu_{test} - x_{true}) / \sigma_{test}$ will give you a sort of 'relativized error'. You can also apply standard statistical tests for significance, e.g. the t-test.

Let me share one approach that makes sense to use in some cases.

In my case, the signal follows roughly the inverse square law in magnitude, but also goes above and below zero, crossing zero at various points. I am interested in the relative error (i.e. far away, where the signal is microvolts, I need precision down to the nanovolt, but near the source, where the signal is a few volts, I need millivolt precision, and would like to ignore deviations in the nanovolt range; so using absolute error doesn't make sense).

But, if I simply divide, either by the true signal, the approximation, or various combinations of the two, the relative error shoots to infinity near the zero-crossings.

The solution is to weigh the absolute error by the inverse of a yardstick signal, that has a similar fall-off properties to the signals of interest, and is positive everywhere. In the formula for relative error, the true signal itself is used for that, but it doesn't have to be, to produce the behaviour you expect from the relative error.

In fact, the normalising signal could be wrong by a multiplicative factor (e.g. if your space is anisotropic, but you still use 1/r^2 as the denominator), and the ratio would still work well as a relative error. Thinking in terms of a log scale helps somewhat, because the relative error becomes a subtraction, rather than division.

EDIT

To quote an article (1) with 600+ citations reported by Google Scholar, from an authority in these numerical issues:

$\epsilon = (f_2 - f_1) / f_1\;\;\;$ (7)

[...]

$E_1$ may of course be expressed as a percent, and like any relative error indicator it will become meaningless when $f_1$ [...] is zero or small relative to $(f_2 - f_1)$, in which case the denominator of Eq. (7) should be replaced with some suitable normalizing value for the problem at hand [...]

(Note that $E_1$ is defined to be a multiple of $\epsilon$ in the article, but these details are irrelevant in the present context.)

I take this as a strong indicator that up until at least 1994, there was no better analogue of relative error for signals that cross zero, than the idea being proposed here, namely, dividing by a normalising signal (that I call the "yardstick" signal above).

I have obtained the best results by applying the Symmetric Mean Absolute Percentage Error (SMAPE) in its third definition:

$$SMAPE (\%)= \frac{\sum_{i=1}^n{\vert x_{true,i} - x_{test,i}\rvert}}{\sum_{i=1}^n{( x_{true,i} + x_{test,i})}}\cdot 100$$

• This is exactly what was proposed by @ClaudeLeibovici in his (accepted) answer, and in a comment under the question. – Alex M. Oct 29 '17 at 11:24

How about $$\frac{|x_\text{true}-x_\text{test}|}{\mu_\text{true}}$$ This would give a percentage relative error of the predicted data in respect to the mean of the real data.