ClassCut for Unsupervised Class Segmentation V 1.0
===================================================
B. Alexe, T. Deselaers, V. Ferrari
{bogdan, deselaers, ferrari}@vision.ee.ethz.ch
This software was developed under Linux with Matlab R2010a (version 7.10) on a
64 bit machine. There is no guarantee it will run on other operating systems
or Matlab versions (though it probably will).
As sub-components, this software uses:
- the objectness measure of [2]
- the segmentation code of [3]
- the edge detector code of [4]
- the k-means implementation and energy minimization tools (TRW-S and QPBO)
from the Visual Geometry Group of the University of Oxford,
http://www.robots.ox.ac.uk/~vgg/ [5].
We thank the authors for permission to redistribute their code as part of
this release.
Introduction
------------
Welcome to this release of ClassCut [1]. ClassCut is a method for unsupervised
class segmentation. Its goal is to jointly segment objects of the same unknown
class from a set of images. ClassCut is based on a binary pairwise energy
function similar to those used in interactive/supervised segmentation, but as
opposed to those, its energy function is defined over a set of images rather
than on a single image. ClassCut alternates two stages: (1) learning/ updating
a class model given the current segmentations; (2) jointly segmenting the
objects in all images given the current class model. It converges when the
segmentation is unchanged in two consecutive iterations.
Quick start
-----------
Let
be the directory where you uncompressed the release archive.
cd
cd ThirdParties/objectness-release/segment
make # compiles the segmentation tool [3]
cd ../../BSE-1.2/
ln -s Makefile.gcc Makefile
make # compiles the edge detector tool [4]
cd
matlab
make # compiles the necessary MEX files for [1], [2] and [5]
Setting things up
-----------------
The only mandatory operations are:
1. add the code to the Matlab path (this is carried out in startup.m)
2. compile the third parties tools (steps 2-9 in 'Quick start');
Another optional operation (recommended):
3. the variable dir_root in runClassCut.m is set to `pwd` to work directly
as detailed in 'Quick start'. We recommend to move your data elsewhere
and update dir_root accordingly. In this way the program will work
regardless of the current working directory.
Matlab Functions
----------------------
runClassCut
-----------------
resultStruct = runClassCut(inputDir, imageType, [weights, params]);
runs ClassCut on the images in inputDir. The resulting segmentations and the learned
class models will be stored in resultStruct.
Input:
inputDir - input directory containing images with instances of the
same class ;
imageType - image type: 'png', 'jpg', 'jpeg', etc.
weights - vector of size 1 x 9 containing weights associated to the
model cues.
The default weights are [1 4 0.2 50 25 10 5 5 0.01];
params - parameters used by ClassCut (loaded during the startup).
Output:
resultStruct - matlab structure containing the segmentations and learned
class models.
Example:
--------
In order to check that the code is working properly try to run ClassCut for the
images provided in the 'Example' directory:
resultStruct = runClassCut([ '/Example/'], 'png');
The output resultStruct, is a matlab structure with the following fields:
-segmentations - cell array containing segmentations of the
corresponding initial images
-location_model - 32x32 array corresponding to the location model in [1]
-shape_model - 32x32x5 array corresponding to the shape model in [1]
-segm_dir - name of the directory containing all the segmentations images
Setting the weights vector
---------------------------
The weights vector is of size 1 x 9 and contains the weights associated to the model cues:
weights = [w1 w2 w3 w4 w5 w6 w7 w8 w9], where
w1 - weight assigned for the appearance cue BOW using 100 visual words
w2 - weight assigned for the appearance cue BOW using 1000 visual words
w3 - weight assigned for the within image smoothness cue
w4 - weight assigned for the border penalty cue
w5 - weight assigned for the area reward cue
w6 - weight assigned for the between image smoothness cue
w7 - weight assigned for the location cue
w8 - weight assigned for the shape cue
w9 - weight assigned for the appearance cue color distributions
For all the experiments reported in [1], the weights are found using a grid-search
procedure (for the exact setup please refer to [1]). We provide all the weights used
for the experiments reported [1] in the 'Weights' directory .
Precomputed coordinate frames from [1]
-------------------------------------------------
This release includes all the coordinate frames used in the experiments carried out in [1].
These coordinate frames are located in the directory 'CoordinateFrames'. In order to be used,
the coordinate frames should be copied in the same directory where the input images are.
Support
-------
For any query/suggestion/complaint or simply to say you like/use this software, just drop us
an email:
bogdan@vision.ee.ethz.ch (please contact this address first)
deselaers@vision.ee.ethz.ch
ferrari@vision.ee.ethz.ch
We wish you a good time using this software,
Bogdan Alexe
Thomas Deselaers
Vittorio Ferrari
References
----------
[1] Bogdan Alexe, Thomas Deselaers and Vittorio Ferrari
ClassCut for Unsupervised Class Segmentation,
ECCV 2010, Heraklion, Greece.
[2] Bogdan Alexe, Thomas Deselaers and Vittorio Ferrari
What is an object?,
CVPR 2010, San Francisco, USA
[3] P. F. Felzenszwalb and D. P. Huttenlocher
Efficient graph-based image segmentation,
IJCV, 2004
http://people.cs.uchicago.edu/~pff/segment/
[4] D. Martin, C. Fowlkes and J. Malik
Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues,
PAMI 26 (5) p.530-549
http://www.cs.berkeley.edu/~fowlkes/BSE/
[5] Visual Geometry Group of the University of Oxford,
http://www.robots.ox.ac.uk/~vgg/
Versions history
----------------
1.0
---
- First public release
0.9
---
- First semi-internal release for testers