1
$\begingroup$

Given the following formula:

Precision = TP / (TP + FP)
Recall = TPR (True Positive Rate)
 F1 = 2((PRE * REC)/(PRE + REC))

What is the correct interpretation for f1-score when precision is Nan and recall isn't?

$\endgroup$
2
  • $\begingroup$ What language is this in? In Python, for example, you can't get NaN by dividing by zero, as it raises an error instead. $\endgroup$
    – J.G.
    Commented Nov 24, 2019 at 17:31
  • $\begingroup$ @J.g It's in c++ $\endgroup$
    – nz_21
    Commented Nov 24, 2019 at 17:32

2 Answers 2

1
$\begingroup$

Presumably the precision $TP/(TP+FP)$ is coming as NaN because $TP=0$ and $FP=0$:

  • If there are no false negatives either, i.e. $FN = 0$ too and the sensitivity is also NaN, then the $F_1$ score really is undefined: there were no cases where your model had any opportunity to identify any true positives whether it was good or not, though at least it had no false negatives. Sine the $F_1$ score ignores true negatives and treats them as irrelevant (so it is not really a complete accuracy measure), you have no useful data for calculation of $F_1$.

  • If there are any false negatives, i.e. $FN \gt 0$, then you can take the $F_1$ score to be zero: your model failed to identify any true positives when it could have done. The way to see this is to note that you can write $F_1 = \frac{2\, TP}{(TP+FP) +(TP + FN)}$.

  • You can use the same argument as the second point to deal with the case where the sensitivity is NaN, i.e. $TP=0$ and $FN=0$ but the precision is $0$ because $FP>0$. Your model identified positives when there was no possibility of true positives, so you can take the $F_1$ score to be zero.

$\endgroup$
1
$\begingroup$

You want to interpret $2TP/(2TP+FP+FN)$ if $TP/(TP+FP)$ is NaN but $TP/(TP+FN)$ isn't. Programming languages come in two kinds for this:

Case I: NaN results from one of the variables being NaN. By process of elimination, it's FP. Therefore, $2TP/(2TP+FP+FN)$ will also be NaN.

Case II: NaN results from a division by $0$. Then $TP=FP=0\ne FN$, so $2TP/(2TP+FP+FN)=0$.

According to these answers, C++ is case II, obtaining NaN from $0/0$.

$\endgroup$
1
  • 2
    $\begingroup$ i'm more concerned about the statistical interpretation, rather than the programming semantics $\endgroup$
    – nz_21
    Commented Nov 24, 2019 at 17:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .