As we know, faster-RCNN has two main parts:one is region proposal network (RPN),and another one is fast-RCNN.

My question is, now that region proposal network (RPN) can output class scores and bounding boxes and is trainable,why do we need Fast-RCNN.

Am I thinking it right that the RPN is enough for detection (red circle),and Fast-RCNN is now becoming redundant (blue circle) ?

  • 1
    $\begingroup$ In my opinion, faster rcnn is an enhancement to RCNN and Fast RCNN, so it maintains the architecture (a region proposal and classifier). I think you asked a good question, rpn might be enough for detection. I think YOLO and SSD are inspired by this idea. $\endgroup$ – Mingjiang Shi Jul 14 '19 at 15:42

Object proposal methods were classically adopted as external modules independent of the detectors. Widely used object proposal methods include those based on grouping super-pixels (e.g., Selective Search) and those based on sliding windows (e.g., objectness in windows).

In Faster RCNN the RPN Network is an objectness based region proposal network that is integrated with the CNN network.

In the original RCNN method and Fast-RCNN, Selective Search is used to generate region proposals while the CNN part is used to classify them into object categories or background independently. Thus the CNN part mainly plays as a classifier, and it does not predict object bounds (except for refining by a bounding box regression layer). Therefore its accuracy depends on the performance of the region proposal module.

The contribution of Faster-RCNN on the other hand, is that the authors proposed a unified way to accomplish both tasks. A simplified way to think of this is that they replaced Selective Search by adopting an RPN. The RPN shared convolutional features with a CNN object detector. Thus the classification loss of the RPN network is used for a binary classification task with the following classes: object, not object, while the classification loss of the CNN is used for classification of object category.

The experiments they performed showed that when the two networks share features (instead of using separate RPN and CNN networks), detection performance is improved.

For more information please check the following resources:




| cite | improve this answer | |
  • $\begingroup$ High, why is the binary RPN classification task using a size two vector, separate number for each of object, not object, instead of just a single probability with a 50% threshold? $\endgroup$ – Austin Jun 8 '18 at 12:04
  • 1
    $\begingroup$ I think the question was why not make RPN also classify directly the objects it finds, question to which this reply doesn't answer $\endgroup$ – Mehdi Oct 19 '18 at 15:13

Main reasons why RPN is not sufficient for detection:

  • Localization: Typically, in detection, one is interested in an accurate localization of the object in an image. RPN uses "anchor" boxes with fixed ratios and scales in a sliding-window fashion to propose regions, which means that the RPN network considers the region likely to contain the object, but does not accuratly determine an accurate location. You still need addional layers to find a more accurate bounding box for the object.

  • Multiple specific classes: Typically, in 'detection', one is interested in localizing as well as classifying a given object. The RPN cannot output arbitrary classes of choice, but rather outpurs an "objectness" score, which indicates the likelihood of the "anchor" box region to contain an object, as opposed to a background. You still need addional layers (i.e. Fast RCNN) to classify the specific object (i.e. as car, person, etc.)

Now if one did not care about finding an accurate bounding box for localization (satisfied with fixed "anchor" boxes), and one did not care about finding specific object classes (satisfied with 'object' vs 'backround'), then the RPN and classification loss (rather 'objectness' loss) could be sufficient to infer regions that are likely to contain an object.

| cite | improve this answer | |
  • $\begingroup$ RPN containes a bounding box regression layer which is trained on the loss to the ground truth box, what is the purpose of having it then? $\endgroup$ – Mehdi Oct 19 '18 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.