University of Zagreb
Faculty of Electrical Engineering and Computing
Unska 3, Zagreb, Croatia

Summary: SCface is a database of static images of human faces. Images were taken in uncontrolled indoor environment using five video surveillance cameras of various qualities. Database contains 4160 static images (in visible and infrared spectrum) of 130 subjects. Images from different quality cameras mimic the real-world conditions and enable robust face recognition algorithms testing, emphasizing different law enforcement and surveillance use case scenarios. SCface database is available to research community through the procedure described below.

IMPORTANT: All documents and papers that report research results obtained
using
the SCface database shoud cite the above paper.

Here is a short summary of what makes this database interesting to face recognition research community:

Different quality and resolution cameras were used;

Images were taken under uncontrolled illumination conditions;

Images were taken from various distances;

Head pose in surveillance images is typical for a commercial surveillance system, i.e. the camera is placed slightly above the subject's head, making the recognition even more demanding; besides, during the surveillance camera recordings the individuals were not looking to a fixed point;

Database contains images of 130 subjects, enough to eliminate performance results obtained by pure coincidence (chances of recognition by pure coincidence is less than 0.7 %);

Both identification and verification scenarios are possible.

SCface database may be obtained free of charge.

The researcher requesting SCface database should include:

A cover letter formally requesting the database. The letter should be written on the requesting institution's own letterhead, and must include a legible address and telephone number. Inclusion of an e-mail address is recommended.

A properly completed SCface Database Release Agreement. Please download it, read it carefully, and complete it appropriately. We require that you sign SCface Database Release Agreement governing the terms of use prior to receipt of the data. Note that the SCface Database Release Agreement should be signed by a full-time staff member (that is, student is not acceptable).

A cover letter and completed SCface Database Release Agreement should be scanned and sent to Prof. Mislav Grgic by e-mail. / You can construct the e-mail address by taking the first name, followed by a dot, followed by the second name (surname), followed by the @ symbol, followed by fer, followed by a dot, followed by hr /

DATABASE DESCRIPTION

SCface database was designed mainly as a means of testing face recognition algorithms in real-world conditions. In such a setup, one can easily imagine a scenario where an individual should be recognized comparing one frontal mug shot image to a low quality video surveillance still image. In order to achieve a realistic setup we decided to use commercially available surveillance cameras of varying quality. Images in other currently available databases are usually taken by the same camera and are not taken using proper, commercially available, surveillance equipment. Since two of our surveillance cameras record both visible spectrum and IR night vision images, we decided to include IR imagery in the database as well.

Image acquisition: equipment setup and imaging procedure

Capturing face images took place in Video Communications Laboratory at the Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia. Capture equipment included: six surveillance cameras, professional digital video surveillance recorder, professional high-quality photo camera and a computer. For mug shot image acquisition we used a high-quality photo camera. For surveillance camera image acquisition we used five different (commercially available) models of surveillance cameras and for IR mug shots a separate surveillance camera was used.

The only source of illumination was the outdoor light, which came through a window on one side. Two (out of five) surveillance cameras were able to record in IR night vision mode as well. The sixth camera was installed in a separate, darkened room for capturing IR mug shots. The high-quality photo camera for capturing visible light mug shots was installed the same way as the infrared camera but in a separate room with the standard indoor lighting and it was equipped with adequate flash. Mug shot imaging conditions are exactly the same as would be expected for any law enforcement or national security use (passport images or any other personal identification document). All six cameras (five surveillance and one IR mug shot) were connected to a professional digital video surveillance recorder, which was recording all six video streams simultaneously all the time on internal hard disk.

The surveillance cameras are named cam1, cam2, cam3, cam4 and cam5. Cam1 and cam5 are also able to work in IR night vision mode. We decided to name the images taken by them in the IR night vision mode as cam6 (actually cam1 in night vision mode) and cam7 (actually cam5 in night vision mode). Camera for taking IR mug shots was named cam8. All cameras (surveillance and photo) were installed and fixed to same positions and were not moved during the whole capturing process.

All participants in this project have passed through the following procedure. First they had to walk in front of the surveillance cameras in the dark and after that they had to do the same in uncontrolled indoor lighting. During their walk in front of the cameras they had to stop at three previously marked positions. This way 21 images per subject were taken (cam1-7 at distances of 4.20, 2.60 and 1.00 meters).

After that, participants were photographed with digital photographer's camera at close range in controlled conditions (standard indoor lighting, adequate use of flash to avoid shades, high resolution of images). This set of images provides nine discrete views of each face, ranging from left to right profile in equal steps of 22.5 degrees. To assure comparable views for each subject, numbered markers were used as fixation points. As a final result, there are nine images per subject with views from -90 to +90 degrees including another mug shot at 0 degrees.

In the end, subjects went into the dark room where high quality IR night vision surveillance camera was installed for capturing IR mug shots at close range. In overall that gives 32 images per subject in the database.

Naming conventions and final database forming

After the capturing procedure subjects' faces were extracted from the gathered images. Captured images were named in the following manner:

Surveillance cameras (cam1-7): subjectID_camNum_distanceLabel.jpg

IR frontal mug shot: subjectID_cam8.jpg

Visible light mug shot: subjectID_frontal.jpg

Different pose images: subjectID_angleLabel.jpg

This way every image in the database gained a unique name carrying information both about a subject's unique ordinal and at what distance and imaging conditions is the image taken. Distance labels 1, 2 and 3 represent distances of 4.20, 2.60 and 1.00 meters, respectively. As an example, the filename 001_cam1_1.jpg means that this image shows subject 001 captured with surveillance camera 1 at a distance of 4.20 meters. The whole surveillance image set (including two mug shots) for one subject is presented below.

001_cam1_1

001_cam1_2

001_cam1_3

001_cam2_1

001_cam2_2

001_cam2_3

001_cam3_1

001_cam3_2

001_cam3_3

001_cam4_1

001_cam4_2

001_cam4_3

001_cam5_1

001_cam5_2

001_cam5_3

001_cam6_1

001_cam6_2

001_cam6_3

001_cam7_1

001_cam7_2

001_cam7_3

001_frontal

001_cam8

Fig. 1. Example of one image set for one subject
(click on each image to see original)

001_L4

001_L3

001_L2

001_L1

001_F

001_R1

001_R2

001_R3

001_R4

Fig. 2. Example of different pose images
(click on each image to see original)

A textual database with coordinates of eyes, tip of the nose and center of the mouth is created, containing the details for all images in the database (except for different pose images).

Fig. 3. Coordinates of the centers of the eyes, tip of the nose and
mouth, and its corresponding textual database entries
(click here or on the above image to see the complete textual database)

DATABASE DEMOGRAPHICS

The participants in this project were students, professors or employees at the Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia. From total of 130 volunteers, 115 were males and 15 females. All participants were Caucasians, between the ages of 20 and 75.

Besides, the textual file containing each subjects date and year of birth will be distributed with the database. This also makes our SCface database unique, because subjects' birthdays are not available in any other database (www.face-rec.org/databases). Moreover, the textual file contains the following additional information: gender, beard, moustache, glasses.

POTENTIAL USES OF THE DATABASE

The first and the most important potential use of this database is to test the face recognition algorithm's robustness in a real-world surveillance scenario. In such a setup a face recognition system should recognize a person by comparing an image captured by a surveillance camera to the image stored in a database. If we postulate a law enforcement use as the most potential scenario, this database image, to which the surveillance image is compared to, is a high quality full frontal facial mug shot. We would specifically like to encourage researchers to explore more deeply the small sample size problem (more dimensions than examples). Having one frontal mug shot per subject in our database addresses this issue adequately.

Our surveillance cameras were of various quality and resolution and this issue is the strong point of this database. It remains to be seen how face recognition algorithms will perform in such difficult conditions and how does the quality of capturing equipment and subject's distance from camera influence the results. There is also a potential to test various image preprocessing algorithms (enhancing image quality by filtering techniques), as some of these surveillance images are of extremely low quality and resolution.

By including different pose images of subjects, we made it possible to use this database in face modeling and 3D face recognition.

This should be a straightforward test comparing the mug shot image (visible light, i.e. subjectID_frontal.jpg images) to cam1–5 images. Frontal mug shots represent the gallery of known images and cam1–5 images and three distances are probe sets (the main idea is similar to FRVT tests). This scenario gives 15 possible different probe sets, varying both in distances from camera and in camera qualities. Comparing the probe image to one gallery image is the most logical real-world (law enforcement) scenario.

NightTime tests have two possible galleries that can be used. Cam6-7 images (IR night vision) can be compared to mug shot images (visible light, i.e. subjectID_frontal.jpg images) or they can be compared to cam8 (IR night vision mug shot) images. This scenario gives two galleries and six possible different probe sets, again varying both in distances from camera and in camera qualities.

This protocol for the SCface database was developed by Mitchell McLaren (The Netherlands) and Roy Wallace (Switzerland). This protocol defines a face authentication/verification scenario, to allow for testing of face verification algorithms in challenging real-world conditions, with substantial lighting, pose and image quality variation. The protocol is based based on the DayTime tests scenario (see above), and utilises images from all 130 subjects in the database. It includes world, development and test sets each with a disjoint set of subjects. Using this structure, the world set is used to background models while the development set is used to tune hyperparameters and select a decision threshold. Finally, the determined threshold is applied to scores from the test set to determine how well the system generalises to unseen subjects.

The authors would like to thank Bozidar Klimpak for his technical support during database acquistion and indexing, Kresimir Marusic and Tehnozavod Marusic Ltd. for providing surveillance system equipment, Boris Krzic for providing professional photo equipment and his help with mug shots capturing, Darko Poljak for developing and providing JAVA software used for semiautomatic determination of eyes, nose and mouth coordinates, and to all participants in this project. The work described was conducted under the research project "Intelligent Image Features Extraction in Knowledge Discovery Systems" (036-0982560-1643), supported by the Ministry of Science, Education and Sports of the Republic of Croatia.