Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
As we know, faster-RCNN has two main parts:one is region proposal network (RPN),and another one is fast-RCNN.
My question is, now that region proposal network (RPN) can output class scores and bounding boxes and is trainable,why do we need Fast-RCNN.
Am I thinking it right that the RPN is enough for detection (red circle),and Fast-RCNN is now becoming redundant (blue circle) ?
Object proposal methods were classically adopted as external modules independent of the detectors. Widely used object proposal methods include those based on grouping super-pixels (e.g., Selective Search) and those based on sliding windows (e.g., objectness in windows).
In Faster RCNN the RPN Network is an objectness based region proposal network that is integrated with the CNN network.
In the original RCNN method and Fast-RCNN, Selective Search is used to generate region proposals while the CNN part is used to classify them into object categories or background independently. Thus the CNN part mainly plays as a classifier, and it does not predict object bounds (except for refining by a bounding box regression layer). Therefore its accuracy depends on the performance of the region proposal module.
The contribution of Faster-RCNN on the other hand, is that the authors proposed a unified way to accomplish both tasks. A simplified way to think of this is that they replaced Selective Search by adopting an RPN. The RPN shared convolutional features with a CNN object detector. Thus the classification loss of the RPN network is used for a binary classification task with the following classes: object, not object, while the classification loss of the CNN is used for classification of object category.
The experiments they performed showed that when the two networks share features (instead of using separate RPN and CNN networks), detection performance is improved.
For more information please check the following resources:
https://www.kdnuggets.com/2017/10/deep-learning-object-detection-comprehensive-review.html
Main reasons why RPN is not sufficient for detection:
Localization: Typically, in detection, one is interested in an accurate localization of the object in an image. RPN uses "anchor" boxes with fixed ratios and scales in a sliding-window fashion to propose regions, which means that the RPN network considers the region likely to contain the object, but does not accuratly determine an accurate location. You still need addional layers to find a more accurate bounding box for the object.
Multiple specific classes: Typically, in 'detection', one is interested in localizing as well as classifying a given object. The RPN cannot output arbitrary classes of choice, but rather outpurs an "objectness" score, which indicates the likelihood of the "anchor" box region to contain an object, as opposed to a background. You still need addional layers (i.e. Fast RCNN) to classify the specific object (i.e. as car, person, etc.)
Now if one did not care about finding an accurate bounding box for localization (satisfied with fixed "anchor" boxes), and one did not care about finding specific object classes (satisfied with 'object' vs 'backround'), then the RPN and classification loss (rather 'objectness' loss) could be sufficient to infer regions that are likely to contain an object.
