Presentation

Although visible face recognition systems have grown as a major area of research, they are still facing serious challenges when operating in uncontrolled environments. In attempt to overcome these limitations, thermal imagery has been investigated as a promising direction to extend face recognition technology. However, the reduced number of databases acquired in thermal spectrum limits its exploration. Therefore, we introduce this database of face images acquired simultaneously in visible and thermal spectra under various variations: illumination, expression, pose and occlusion.

ACQUISITION PROCESS

The presented database was acquired with a newly developed dual sensor, visible and thermal, camera FLIR Duo R by FLIR Systems. This camera is designed for unmanned aerial vehicle allowing to capture simultaneously images and videos in both visible and thermal spectra. Accordingly, it is well suited for our database collection aimed at fusion for face recognition as well as cross-spectrum applications. The visible sensor is a CCD sensor with a pixel resolution of 1920×1080. The thermal sensor of this camera is an uncooled VoX microbolometer and has a spectral response range of 7.5 - 13.5µm with a pixel resolution of 160×120.

flir

Figure 1: FLIR Duo R

setup

Figure 2: Acquisition setup


STRUCTURE

The database was collected from 50 subjects of different age, sex and ethnicity, resulting a total of 2100 images. The camera was designed to take simultaneous face images of thermal and visible spectra. Before the acquisition process, volunteers were asked to fill and sign consent and metadata forms approved by the CNIL ”Commission nationale de l’informatique et des libertés”. During the data collection, the camera was set to capture a shot every second. Each subject was asked to perform several facial expressions, to change the head pose, to wear some items like sunglasses and cap, and finally we vary the light while the subject stays in a neutral state.

The variations considered during this session are shown in figure 4 and described as follow:

  • Expression: 7 pairs captured with standard illumination, frontal pose with different face expression: neutral, happy, angry, sad, surprised, blinking, yawning.
  • Head pose: 4 pairs captured with standard illumination, neutral expression with different head poses: up, down, right at 45° left at 45°.
  • Occlusion: 5 pairs captured with standard illumination, frontal pose, neutral expression and varying occlusions: eyeglasses, sunglasses, cap, mouth occluded by hand, eye occluded by hand.
  • Illumination: 5 pairs captured with frontal pose, neutral expression and different illuminations: Ambient light, rim light, key light, fill light, all lights on, all lights off.
var
                                 Figure 3: Illustration of visible and thermal images for various facial variations.

NAMING CONVENTION

Files are dispatched into directories according to the subject ID. Each directory contains 2 sub-directories (Visible, Thermal). Each of the 2 directories contains face images of the subjects in the different variations presented above. 

Filenames are constructed according the following grammar :
[Spectrum]_[SubjectID]_[SessionID]_[VariationID]_[VariationACR].[ImageFormat]

  • Spectrum: refers to the spectrum in which the image was acquired, "VIS" for visible spectrum and "TH" for thermal spectrum.
  • SubjectID: is a three digit integer uniquely identifying the subject, starting from 001 to 050.
  • SessionID: is a digit indicating the acquisition session number, notably “1” or “2”, the second session will be uploaded shortly.
  • VariationID: is a two digit integer indicating the variation number, ranging from 01 to 21, corresponding to different variations illustrated in Figure 3.
  • VariationACR: is an acronym identifying the variation in a format more suitable for human reading {NN: neutral, EH: Expression - Happy, EA: Expression - Angry, ES: Ecpression - Sad, ESp: Expression - Surprised, AEC: Action - Eyes Closed, AOM: Action - Open Mouth, PU: Pose - Up, PD: Pose - Down, PL: Pose - Left, PR: Pose - Right, OOG: Occlusion - Optical Glasses, OSG: Occlusion - SunGlasses, OH: Occlusion - Hat, OHM: Occlusion - Hand on Mouth, OHE: Occlusion - Hand on Eye, LLU: Light - Light Up, LLR: Light - Light Right, LLL: Light - Light Left, LD: Light - Dark, LR: Light - Room light}
  • ImageFormat: refers to the format of the images, TIFF for thermal images and JPG for visible images.

The database comes with an excel file containing the following metadata for each subject:

  • Date if the acquisition
  • Age of the subject
  • Gender of the subject
  • If the subject agreed that his/her face images can displayed in scientific articles/presentations

REFERENCE

Any publication using this database must cite the following paper:

K. Mallat, J-L. Dugelay, « A benchmark database of visible and thermal paired face images across multiple variations », International Conference of the Biometrics Special Interest Group BIOSIG, pages 199-206, 2018

@inproceedings{Mallat18,
author = {{M}allat, {K}hawla and  {D}ugelay, {J}ean-{L}uc},
title     = {A benchmark database of visible and thermal paired face images across multiple variations},
booktitle = {International Conference of the Biometrics Special Interest Group,   {BIOSIG} 2018, Darmstadt, Germany, September},
series = {{LNI}},
publisher = {{GI} / {IEEE}},
pages = {199 - 206}
year   = {2018},
}