I. CALVIN Upper Body Detector v1.04
Software by M. Eichner and V. Ferrari
based on:
- the detection framework of Felzenszwalb et al. [4]
- training data by M. Marin et al. [3]
- the OpenCV implementation [6] of the face detector [5]
requirements:
- you must first install the detection framework [4] (voc-release-3.1); a snapshot is available on the same webpage as this Upper Body detector
- (optional) for best performance install also the OpenCV software package [6]; this Upper Body detector will then integrate also a face detector and will detect more people; (tested with OpenCV v2.0, earlier versions should also be compatible).
This software was developed under Linux with Matlab R2009a.
There is no guarantee it will run on other operating systems or Matlab versions.
back to the top
II. Quick start
0. Let <dir> be the directory where you uncompressed the .tgz release archive.
1. If you want to benefit from the complementary OpenCV face detector make sure that the OpenCV shared libraries
are reachable through the LD_LIBRARY_PATH environment variable:
e.g.
from the bash command line execute:
LD_LIBRARY_PATH="path_to_opencv_lib:$LD_LIBRARY_PATH"
2. Start Matlab
3. Execute the following Matlab commands:
cd <dir>/code
addpath(pwd);
load('detenv.mat');
addpath(absolute_path_to_voc-release-3.1); % voc-release-3.1 is the release of the detection framework [4]
4. If you are running the code for the first time and you want to benefit from the complementary face detector then execute:
mex -Lpath_to_opencv_lib_dir -Ipath_to_opencv_include_dir -lcv -lcxcore me_HaarDetectOpenCV.cpp
5. The Matlab environment is now ready for running our upper body detector. To run it on the provided example execute:
[ubfdetections] = DetectStillImage('../example_data/images/000000.jpg','pff_model_upperbody_final.mat','haarcascade_frontalface_alt2.xml',det_pars,2)
A warning message is displayed if me_HaarDetectOpenCV was not successfully mexed at step 4.
If you do not want to use the complementary face detector replace the 'haarcascade_frontalface_alt2.xml' with [].
This will produce a figure showing the example image with detection bounding-boxes overlaid.
If this output matches exactly <dir>/000000_detections.jpg, then the detector is working perfectly.
Note that the result is reproducible exactly only if you have set up the system completely, i.e. including the face detector.
Otherwise the number of detections will be lower.
back to the top
III. Introduction
We release here software for human upper body detection in still images.
This release is a more portable and better performing replacement for the detector in [3].
It is based on the successful part-based object detection framework [4]. This release contains a model to detect near-frontal upper-bodies,
trained from the data of [3]. The resulting detector returns bounding-boxes fitting the head and upper half of the torso of the person, as the detector [3].
While our detector is useful in general, it is especially tailored to be used within our human pose estimation framework [1,2].
For example, we use the boxes of [4] but we ensure a fixed aspect-ratio by recomputing the horizontal coordinates of the boxes
according to the predefined height-to-width aspect-ratio value (0.9).
The bounding-boxes returned by the detector released here can be directly fed into our pose estimation software [1].
For the comfort of the user, [1] includes a matlab routine to easily interface with the detector released here.
By installing both this release and [1] the user gets a complete, fully automatic, ready-to-run human detection and pose estimation pipeline.
In order to find more people we complement the primary upper-body detections with the upper-body detections regressed from the
Viola-Jones [5] face detector. This is especially valuable for people in poses difficult to detect by the upper-body model (e.g. arms raised above the head).
We use the face detector implementation available in OpenCV [6] together with one of the pre-trained face models.
For this release we estimated regression parameters from a small set of images to map the face detections into detections covering exactly the same portion
of a person as the primary upper-body detector.
Overall, the upper-body detector algorithm we release here is:
1. run the primary upper-body detector based on [4]
2. run the face detector [5,6]
3. regress face detections to obtain secondary upper-body detections
4. run non-maximal suppression on both primary and secondary upper-body detections.
During this step we assign -Inf score to all secondary detections so they are only used to complement to the primary detector.
In a summary, this release improves over [3] in that:
+ it is easily portable to any Linux platform, as it is based on the excellent code of [4]
+ the primary upper-body detector based on [4] performs better than the one in [3] based on [7] (less false-positives)
+ thanks to the complementary face detector, more people are found than in [3]
back to the top
IV. Provided models
pff_model_upperbody_final.mat
model used in the object detection framework [4], trained on the upper-body dataset of [3].
haarcascade_frontalface_alt2.xml
copy of a face model publicly available in OpenCV [6]
back to the top
V. Matlab functions
DetectStillImage
------------------
bbox = DetectStillImage(fullimgpath,ubfmodel_path,facemodel_path,det_pars,verbose);
runs the upper-body detector and optionally the face-detector on an image.
Face detections are automatically regressed to upper-body detections using pre-learned parameters
Input:
fullimgpath - relative/absolute path to an image
pffubfmodel_path - relative/absolute path to the pretrained upper body part-based model
facemodel_path - (optional) relative/absolute path to the pretrained opencv face model (xml file)
if [] then skip face detection
det_pars.iou_thresh - Intersection over Union threshold used during non-maximal suppression
.opencv_face_regparams - [p1 p2 p3 p4] - opencv_face detection to upperbody detection regression parameters
.ubfpff_scale - scale at which [4] is run; higher -> detecting smaller people
verbose - 0 - no output
- 1 - print on screen
- 2 - show images
Output:
bbox(i,:)= [x1 y1 x2 y2 score] set of detections
DetectDir
----------
[detections Files] = DetectDir(img_dir,pffubfmodel_path,facemodel_path,det_pars,verbose)
executes DetectStillImage on a directory with images
Input:
img_dir - relative/absolute path to an directory with images
pffubfmodel_path - relative/absolute path to the pretrained upper body part-based model
facemodel_path - (optional) relative/absolute path to the pretrained opencv face model (xml file)
if [] then skip face detection
det_pars.iou_thresh - Intersection over Union threshold used during non-maximal suppression
.opencv_face_regparams - [p1 p2 p3 p4] - opencv_face detection to upperbody detection regression parameters
.ubfpff_scale - scale at which [4] is run; higher -> detecting smaller people
verbose - 0 - no output
- 1 - print on screen
- 2 - show images
Output:
detections - detection results for a directory, detections{i} - detections found in Files(i).name
Files - struct array as return by dir matlab function for all processed images.
routine creates also a sub directory 'dets' where results are saved
back to the top
VI. Support
For support please contact us:
eichner@vision.ee.ethz.ch
ferrari@vision.ee.ethz.ch
Have fun!
back to the top
VII. References
[1] M.Eichner, V.Ferrari
Human Pose Estimation in Still Images V 1.05
http://www.vision.ee.ethz.ch/~calvin/articulated_human_pose_estimation_code/
[2] M.Eichner, V.Ferrari
Better appearance models for pictorial structures
BMVC 2009
[3] M. Marin, V. Ferrari, A. Zisserman:
upper-body detector
www.robots.ox.ac.uk/~vgg/software/UpperBody/index.html
[4] P. Felzenszwalb, R. Girshick, D. McAllester, D. Ramanan
Object Detection with Discriminatively Trained Part Based Models
PAMI 2009
[5] P. Viola, M. Jones
Rapid Object Detection using a Boosted Cascade of Simple Features
CVPR 2001
[6] http://opencv.willowgarage.com/wiki/
[7] N. Dalal and B. Triggs
Histograms of Oriented Gradients for Human Detection
CVPR 2005
back to the top
VIII. Version history
1.04
----
- a memory leak fixed, noticable when processing many images with no detections; thanks to Huizhong Chen for pointing that out
1.03
----
- optionally runs [4] on an upscaled image resulting in detections of persons at low resolution
1.02
----
- adjusted detector parameters for better detection rate, and less false positives
- improved accuracy of person localization by using the boxes of [4] refined by the
model parts and after setting their aspect-ratio to constant (v1.01 instead used [4]'s root boxes)
1.01
----
- DetectDir stores images with the detection overlays
1.0
---
- initial release
back to the top