The presented database was acquired with a newly developed dual sensor, visible and thermal, camera FLIR Duo R by FLIR Systems. This camera is designed for unmanned aerial vehicle allowing to capture simultaneously images and videos in both visible and thermal spectra. Accordingly, it is well suited for our database collection aimed at fusion for face recognition as well as cross-spectrum applications. The visible sensor is a CCD sensor with a pixel resolution of 1920×1080. The thermal sensor of this camera is an uncooled VoX microbolometer and has a spectral response range of 7.5 - 13.5µm with a pixel resolution of 160×120.

The database was collected from 50 subjects of different age, sex and ethnicity, resulting a total of 2100 images. The camera was designed to take simultaneous face images of thermal and visible spectra. Before the acquisition process, volunteers were asked to fill and sign consent and metadata forms approved by the CNIL ”Commission nationale de l’informatique et des libertés”. During the data collection, the camera was set to capture a shot every second. Each subject was asked to perform several facial expressions, to change the head pose, to wear some items like sunglasses and cap, and finally we vary the light while the subject stays in a neutral state.

The variations considered during this session are shown in figure 4 and described as follow:

• Expression: 7 pairs captured with standard illumination, frontal pose with different face expression: neutral, happy, angry, sad, surprised, blinking, yawning.
• Head pose: 4 pairs captured with standard illumination, neutral expression with different head poses: up, down, right at 45° left at 45°.
• Occlusion: 5 pairs captured with standard illumination, frontal pose, neutral expression and varying occlusions: eyeglasses, sunglasses, cap, mouth occluded by hand, eye occluded by hand.
• Illumination: 5 pairs captured with frontal pose, neutral expression and different illuminations: Ambient light, rim light, key light, fill light, all lights on, all lights off.

Files are dispatched into directories according to the subject ID. Each directory contains 2 sub-directories (Visible, Thermal). Each of the 2 directories contains face images of the subjects in the different variations presented above.

Filenames are constructed according the following grammar : [Spectrum][SubjectID][SessionID][VariationID][VariationACR].[ImageFormat]

• Spectrum: refers to the spectrum in which the image was acquired, “VIS” for visible spectrum and “TH” for thermal spectrum.
• SubjectID: is a three digit integer uniquely identifying the subject, starting from 001 to 050.
• SessionID: is a digit indicating the acquisition session number, notably “1” or “2”, the second session will be uploaded shortly.
• VariationID: is a two digit integer indicating the variation number, ranging from 01 to 21, corresponding to different variations illustrated in Figure 3.
• VariationACR: is an acronym identifying the variation in a format more suitable for human reading {NN: neutral, EH: Expression - Happy, EA: Expression - Angry, ES: Expression - Sad, ESp: Expression - Surprised, AEC: Action - Eyes Closed, AOM: Action - Open Mouth, PU: Pose - Up, PD: Pose - Down, PL: Pose - Left, PR: Pose - Right, OOG: Occlusion - Optical Glasses, OSG: Occlusion - SunGlasses, OH: Occlusion - Hat, OHM: Occlusion - Hand on Mouth, OHE: Occlusion - Hand on Eye, LLU: Light - Light Up, LLR: Light - Light Right, LLL: Light - Light Left, LD: Light - Dark, LR: Light - Room light}
• ImageFormat: refers to the format of the images, TIFF for thermal images and JPG for visible images.

The database comes with an excel file containing the following metadata for each subject:

• Date if the acquisition
• Age of the subject
• Gender of the subject
• If the subject agreed that his/her face images can displayed in scientific articles/presentations

Any publication using this database must cite the following paper:

K. Mallat, J-L. Dugelay, « A benchmark database of visible and thermal paired face images across multiple variations », International Conference of the Biometrics Special Interest Group BIOSIG, pages 199-206, 2018

@inproceedings{Mallat18,
author = {{M}allat, {K}hawla and {D}ugelay, {J}ean-{L}uc},
title = {A benchmark database of visible and thermal paired face images across multiple variations},
booktitle = {International Conference of the Biometrics Special Interest Group, {BIOSIG} 2018, Darmstadt, Germany, September},
series = {{LNI}},
publisher = {{GI} / {IEEE}},
pages = {199 - 206}
year = {2018},
}

A download link for the database compressed and a password for decrypting the compressed VIS-TH ZIP files will be provided after receiving the duly signed license agreement. Please fill in the license agreement and send a scanned copy by e-mail at vis-th@eurecom.fr