Objectness measure V2.2

Bogdan Alexe, Thomas Deselaers, Vittorio Ferrari

Overview

What is objectness?

The objectness measure acts as a class-generic object detector. It quantifies how likely it is for an image window to contain an object of any class, such as cars and dogs, as opposed to backgrounds, such as grass and water. We release here software for computing objectness [1,2] and sampling any desired number of windows from an image according to their probability of containing an object.

For applications, we recommend to sample about 1000 windows, which ensures covering most objects even in very difficult images (e.g. with small objects and lots of clutter). However, in images of normal difficulty 100 windows are sufficient (e.g. images downloaded from image search engines).

From version V 1.5 we include a new window sampling strategy (NMS) which leads to higher detection rates. On the highly challenging PASCAL VOC 2007 dataset [3], the top 1000 sampled windows now cover 91% of all objects [2], as opposed to about 70% in previous versions using multinomial sampling [1].

In addition to the source code, we also release sampled windows for every image from PASCAL VOC 2007 [3], for both the new NMS sampling strategy and for the older multinomial sampling. These ready-to-use windows hopefully will facilitate applications on this dataset.

How fast is it?

Objectness is computationally efficient. On a mid-range PC, it takes less than 3 seconds to compute the objectness measure and to sample 1000 windows, for an image of size 350 x 500.

Applications of objectness

Objectness is intended as a low-level preprocessing stage, to propose a small number of windows likely to cover all objects in the image. It has been used in several applications so far:

+ weakly supervised localization [4,5,9,12,15,16,31,32,33]

+ weakly supervised segmentation of a single object class [6] and of multiple classes [7]

+ unsupervised object discovery [8,34]

+ content-aware image resizing [10,11]

+ speeding up class-specific detectors [1,2,29]

+ reducing the false-positive rates of class-specific detectors [1,2]

+ object tracking in video [17,21,22,35]

+ large-scale knowledge transfer [18,19,20]

+ co-segmentation [26,27,40]

+ video co-segmentation [39]

+ image quality assessment [28]

+ image difficulty assessment [36]

+ building block or inspiration for other class generic detectors [13,23,24,25,30,37]

+ salient object detection [14, 42]

+ background detection [41]

+ improving spatial support for object detection measurements [38]

+ improving accuracy of object localization at superpixel level [43]

Examples

For each of image we show the windows best covering the objects annotated in the PASCAL VOC 2007 (our of 1000 windows sampled with NMS). We mark in yellow windows correctly covering ground-truth objects (cyan); if there is more than one correct window, the best one is shown.

Objectness maps

For each image we show its pixelwise objectness map. This is obtained by sampling 1000 windows using the NMS sampling procedure and accumulating them. We do this by computing for each image pixel the sum of the objectness scores for the sampled windows containing it. Objectness maps provide meaningful distributions over the object locations, demonstrating that it reduces their uncertainty.

Downloads

Filename	Description	Size
Source code (Matlab/C)	Souce code for objectness measure	21 MB
README.txt	Description of content	10 kB
LICENSE	Software license	1 kB
PASCAL VOC 2007 windows using NMS sampling	1000 windows sampled using the NMS strategy for each image in PASCAL VOC 2007 (recommended)	86 MB
PASCAL VOC 2007 windows using multinomial sampling	10000 windows sampled using the multinomial strategy for each image in PASCAL VOC 2007	576 MB

New in V 2.2
+ minor speedups thanks to avoiding intermediate load/save steps; this software now no longer needs the external tool 'convert' to be installed on your machine

New in V 2.1
+ sampling windows for small sized images doesn't crash anymore

New in V 2.0
+ mex file to fastly compute the superpixel segmentation added
+ function to compute the objectness heat map of an image added

New in V 1.5
+ NMS sampling procedure added

New in V 1.01
+ windows with width or height = 1 pixel are not anymore considered

Publications

[1] Alexe, B., Deselares, T. and Ferrari, V.
What is an object?
CVPR 2010.
Document: PDF

[2] Alexe, B., Deselares, T. and Ferrari, V.
Measuring the objectness of image windows
PAMI 2012.
Document: PDF

[3] Everingham, M., Van Gool, L., Williams, C., Winn, J., and Zisermann, A.
The PASCAL Visual Object Classes Challenge 2007

[4] Deselares, T., Alexe, B. and Ferrari, V.
Localizing objects while learning thier appearance
ECCV 2010.

[5] Khan, I., Roth, P. M. and Bischof, H.
Learning Object Detectors from Weakly-Labeled Internet Images
OAGM Workshop 2011.

[6] Alexe, B., Deselaers, T. and Ferrari, V.
ClassCut for unsupervised class segmentation
ECCV 2010.

[7] Vezhnevets, A., Ferrari, V. and Buhmann, J.
Weakly supervised semantic segmentation with a multi-image model
ICCV 2011.

[8] Lee, Y. J. and Grauman, K.
Learning the easy things first: Self-paced visual category discovery
CVPR 2011.

[9] Prest, A., Schmid, C. and Ferrari, V.
Weakly supervised learning of interactions between humans and objects
PAMI 2011.

[10] Sun, J., and Ling, H.
Scale and Object Aware Image Retargeting for Thumbnail Browsing
ICCV 2011.

[11] Bao, X., Narayan, T., Sani, A. A., Richter, W., Choudhury, R. R., Zhong, L. and Satyanarayanan, M.
The Case for Context-Aware Compression
ACM Hotmobile 2011.

[12] Siva, P. and Xiang, T.
Weakly Supervised Object Detector Learning with Model Drift Detection
ICCV 2011.

[13] Rahtu, E., Kannala, J. and Blaschko, M.
Learning a Category Independent Object Detection Cascade
ICCV 2011.

[14] Chang, K. Y., Liu, T. L., Chen, H. T., and Lai, S. H.
Fusing Generic Objectness and Visual Saliency for Salient Oject Detection
ICCV 2011.

[15] Siva, P., Russell C., and Xing, T.
In Defense of Negative Mining for Annotating Weakly Labelled Data
ECCV 2012.

[16] Sener, F., Bas, C., and Ikizler-Cinbis, N.
On Recognizing Action in Still Images via Multiple Features
1st Workshop on Action Recognition and Pose Estimation in Still Images, ECCV 2012.

[17] Stadler, S., Grabner, H., and Van Gool, L.
Dynamic Objectness for Adaptive Tracking
ACCV 2012.

[18] Guillaumin, M., and Ferrari, V.
Large-scale knowledge transfer for object localization in ImageNet
CVPR 2012.

[19] Kuettel, D., and Ferrari, V.
Figure-ground segmentation by transferring window masks
CVPR 2012.

[20] Kuettel, D., Guillaumin, M., and Ferrari, V.
Segmentation Propagation in ImageNet
ECCV 2012.

[21] Spampinato, C., and Palazzo, S.
Enhancing object detection performance by integrating motion objectness and perceptual organization
ICPR 2012.

[22] Lu, Z., and Grauman, K.
Story-Driven Summarization for Egocentric Video
CVPR 2013.

[23] Blaschko, M., Kannala, J., and Rahtu, E.
Non Maximal Suppression in Cascade Ranking Models
SCIA 2013.

[24] Siva, P., Russell, C., Xiang, T., and Agapito, L.
Looking Beyond the Image: Unsupervised Learning for Object Saliency and Detection
CVPR 2013.

[25] Ristin, M., Gall, J., and Van Gool, L.
Local Context Priors for Object Proposal Generation
ACCV 2012.

[26] Rubio, J., Serrat, J., Lopez, A., and Paragios, N.
Unsupervised co-segmentation through region matching
CVPR 2012.

[27] Meng, F., Li, H., Liu, G., and Ngan, K. N.
Object Co-segmentation based on Shortest Path Algorithm and Saliency Model
OCSPASM 2011.

[28] Mai, L., Le, H., Niu, Y., and Liu, F.
Rule of Thirds Detection from Photograph
ISM 2012.

[29] Saenko, K., Karayev, S., Jia, Y., Shyr, A., Janoch, A., Long, J., Fritz, M., and Dareell, T.
Practical 3-D Object Detection Using Category and Instance-level Appearance Models
IROS 2011.

[30] Gao, Y., Zhang, J., Zhang, L., and Hu, Y.
Finding objects at indoor environment combined with depth information
ICMA 2011.

[31] Shapovalova, N., Vahdat, A., Cannons, K., Lan, T., and Mori, G.
Similarity Constrained Latent Support Vector Machine: An Application to Weakly Supervised Action Classification
ECCV 2012.

[32] Deselaers, T., Alexe, B., and Ferrari, V.
Weakly Supervised Localization and Learning with Generic Knowledge
IJCV 2012.

[33] Prest, A., Leistner, C., Civera, J., Schmid, C., and Ferrari, V.
Learning Object Class Detectors from Weakly Supervised Annotated Video
CVPR 2012.

[34] Bodesheim, P.
Spectral Clustering of ROIs for Object Discovery
DAGM 2011.

[35] Loo, W. and Kim, T.K.
Generic Object Crowd Tracking by Multi-Task Learning
BMVC 2013.

[36] Russakovsky, O., Deng, J., Huang, Z., Berg, A. and Fei-Fei, L.
Detecting avocados to zucchinis: what have we done, and where are we going?
ICCV 2013.

[37] Jia, Y. and Han, M.
Category-Independent Object-level Saliency Detection
ICCV 2013.

[38] Cinbis, R. G., Verbeek, J. and Schmid, C.
Segmentation Driven Object Detection with Fisher Vectors
ICCV 2013.

[39] Rubio, J. C., Serrat, J. and Lopez, A.
Video co-segmentation
ECCV 2012.

[40] Meng, F., Li, H., Ngan, K. N., Zeng, L. and Wu, Q.
Feature Adaptive Co-segmentation by Complexity Awareness
TIP 2013.

[41] Karaoglu, S., Van Gemert, J. C. and Gevers, T.
Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification
ACMMM 2013.

[42] Jiang, P., Ling, H., Yu, J. and Peng, J.
Salient Region Detection by UFO: Uniqueness, Focusness and Objectness
ICCV 2013.

[43] Li, L., Feng, W., Wan, L., and Zhang, J.
Maximum Cohesive Grid of Superpixels for Fast Object Localization
CVPR 2013.

Acknowledgements

This work is funded by the Swiss National Science Foundation SNSF

ETH ZURICH | | CALVIN group