Selective search is a computer vision algorithm utilized to extract multiple high-quality region proposals from an input image. By selecting proposed regions at multiple scales and with varying shapes and sizes, the algorithm provides a set of candidate bounding boxes that can subsequently be evaluated by an object detection model.

Selective Search

The region proposal network (RPN) is an internal component of Faster R-CNN that generates region proposals by processing CNN-extracted feature maps in four steps. First, a $$3 	imes 3$$ convolutional layer with a padding of $$1$$ transforms the feature maps into a new output with $$c$$ channels, providing a length-$$c$$ feature vector for each spatial unit. Second, multiple anchor boxes of varying scales and aspect ratios are generated and centered on each pixel. Third, these length-$$c$$ feature vectors are used to predict a binary class (object or background) and a bounding box for each anchor box. Finally, non-maximum suppression is applied to the bounding boxes predicted as objects to remove overlapping results, yielding the final region proposals required by the region of interest pooling layer.

Region Proposal Network

In object detection, a region proposal is a candidate bounding box extracted from an input image that potentially contains an object of interest. Models typically extract a large set of these proposed regions to process further, evaluating them to predict specific object classes and refine bounding box coordinates. Additionally, predefined anchor boxes can also serve as region proposals.

Claude

Google

The Region-based Convolutional Neural Network (R-CNN) is an object detection model that operates by first extracting a large number of region proposals (e.g., $$2000$$) from an input image. Anchor boxes can also be considered as such proposals. A Convolutional Neural Network (CNN) is then used to perform forward propagation on each individual region proposal to extract its features. Finally, these features are utilized to predict the class and bounding box for each proposed region.

Region-based Convolutional Neural Network

Dive into Deep Learning

The first step in the Region-based Convolutional Neural Network (R-CNN) pipeline is to perform selective search to extract multiple high-quality region proposals from the input image. These regions are typically selected at various scales, shapes, and sizes. Each of these region proposals is then labeled with a ground-truth class and a bounding box to be used in subsequent stages.

Selective Search in R-CNN

Although the Region-based Convolutional Neural Network (R-CNN) model effectively utilizes pretrained Convolutional Neural Networks (CNNs) to extract features, it suffers from significant computational inefficiency. Because the model relies on extracting thousands of region proposals from a single input image, it must execute thousands of separate CNN forward propagations to perform object detection. This massive computational burden makes the standard R-CNN architecture too slow and infeasible for widespread use in real-world applications.

Learn Before

Related

Learn After