ClassCut for Unsupervised Class Segmentation V 1.0 =================================================== B. Alexe, T. Deselaers, V. Ferrari {bogdan, deselaers, ferrari}@vision.ee.ethz.ch This software was developed under Linux with Matlab R2010a (version 7.10) on a 64 bit machine. There is no guarantee it will run on other operating systems or Matlab versions (though it probably will). As sub-components, this software uses: - the objectness measure of [2] - the segmentation code of [3] - the edge detector code of [4] - the k-means implementation and energy minimization tools (TRW-S and QPBO) from the Visual Geometry Group of the University of Oxford, http://www.robots.ox.ac.uk/~vgg/ [5]. We thank the authors for permission to redistribute their code as part of this release. Introduction ------------ Welcome to this release of ClassCut [1]. ClassCut is a method for unsupervised class segmentation. Its goal is to jointly segment objects of the same unknown class from a set of images. ClassCut is based on a binary pairwise energy function similar to those used in interactive/supervised segmentation, but as opposed to those, its energy function is defined over a set of images rather than on a single image. ClassCut alternates two stages: (1) learning/ updating a class model given the current segmentations; (2) jointly segmenting the objects in all images given the current class model. It converges when the segmentation is unchanged in two consecutive iterations. Quick start ----------- Let be the directory where you uncompressed the release archive. cd cd ThirdParties/objectness-release/segment make # compiles the segmentation tool [3] cd ../../BSE-1.2/ ln -s Makefile.gcc Makefile make # compiles the edge detector tool [4] cd matlab make # compiles the necessary MEX files for [1], [2] and [5] Setting things up ----------------- The only mandatory operations are: 1. add the code to the Matlab path (this is carried out in startup.m) 2. compile the third parties tools (steps 2-9 in 'Quick start'); Another optional operation (recommended): 3. the variable dir_root in runClassCut.m is set to `pwd` to work directly as detailed in 'Quick start'. We recommend to move your data elsewhere and update dir_root accordingly. In this way the program will work regardless of the current working directory. Matlab Functions ---------------------- runClassCut ----------------- resultStruct = runClassCut(inputDir, imageType, [weights, params]); runs ClassCut on the images in inputDir. The resulting segmentations and the learned class models will be stored in resultStruct. Input: inputDir - input directory containing images with instances of the same class ; imageType - image type: 'png', 'jpg', 'jpeg', etc. weights - vector of size 1 x 9 containing weights associated to the model cues. The default weights are [1 4 0.2 50 25 10 5 5 0.01]; params - parameters used by ClassCut (loaded during the startup). Output: resultStruct - matlab structure containing the segmentations and learned class models. Example: -------- In order to check that the code is working properly try to run ClassCut for the images provided in the 'Example' directory: resultStruct = runClassCut([ '/Example/'], 'png'); The output resultStruct, is a matlab structure with the following fields: -segmentations - cell array containing segmentations of the corresponding initial images -location_model - 32x32 array corresponding to the location model in [1] -shape_model - 32x32x5 array corresponding to the shape model in [1] -segm_dir - name of the directory containing all the segmentations images Setting the weights vector --------------------------- The weights vector is of size 1 x 9 and contains the weights associated to the model cues: weights = [w1 w2 w3 w4 w5 w6 w7 w8 w9], where w1 - weight assigned for the appearance cue BOW using 100 visual words w2 - weight assigned for the appearance cue BOW using 1000 visual words w3 - weight assigned for the within image smoothness cue w4 - weight assigned for the border penalty cue w5 - weight assigned for the area reward cue w6 - weight assigned for the between image smoothness cue w7 - weight assigned for the location cue w8 - weight assigned for the shape cue w9 - weight assigned for the appearance cue color distributions For all the experiments reported in [1], the weights are found using a grid-search procedure (for the exact setup please refer to [1]). We provide all the weights used for the experiments reported [1] in the 'Weights' directory . Precomputed coordinate frames from [1] ------------------------------------------------- This release includes all the coordinate frames used in the experiments carried out in [1]. These coordinate frames are located in the directory 'CoordinateFrames'. In order to be used, the coordinate frames should be copied in the same directory where the input images are. Support ------- For any query/suggestion/complaint or simply to say you like/use this software, just drop us an email: bogdan@vision.ee.ethz.ch (please contact this address first) deselaers@vision.ee.ethz.ch ferrari@vision.ee.ethz.ch We wish you a good time using this software, Bogdan Alexe Thomas Deselaers Vittorio Ferrari References ---------- [1] Bogdan Alexe, Thomas Deselaers and Vittorio Ferrari ClassCut for Unsupervised Class Segmentation, ECCV 2010, Heraklion, Greece. [2] Bogdan Alexe, Thomas Deselaers and Vittorio Ferrari What is an object?, CVPR 2010, San Francisco, USA [3] P. F. Felzenszwalb and D. P. Huttenlocher Efficient graph-based image segmentation, IJCV, 2004 http://people.cs.uchicago.edu/~pff/segment/ [4] D. Martin, C. Fowlkes and J. Malik Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues, PAMI 26 (5) p.530-549 http://www.cs.berkeley.edu/~fowlkes/BSE/ [5] Visual Geometry Group of the University of Oxford, http://www.robots.ox.ac.uk/~vgg/ Versions history ---------------- 1.0 --- - First public release 0.9 --- - First semi-internal release for testers