Spoofing and Disguise Variations in Face Recognition

Human recognition has become an important topic as the need and investments for security applications grow continuously. Biometrics enable reliable and efficient identity management systems by using physical and behavioral characteristics of the subjects that are permanent, universal and easy to access. This is why, the topic of biometrics attracts higher attention today. Numerous biometric systems exist which utilize various human characteristics. Among all biometrics traits, face recognition is advantageous in terms of accessibility and reliability. It allows identification at relatively high distances for unaware subjects that do not have to cooperate. In this dissertation, two challenges in face recognition are analyzed. The first one is face spoofing. Initially, spoofing in face recognition is explained together with the countermeasure techniques that are proposed for the protection of face recognition systems against spoofing attacks. The second challenge explored in this thesis is disguise variations. In the second part of the thesis, the impact of disguise variations on face recognition is analyzed and the proposed techniques that are robust to these variations are explained. In a spoofing attempt, a person tries to masquerade as another person and thereby, tries to gain access to a recognition system. Since face data can be acquired easily in a contactless manner, spoofing is a real threat for face recognition systems. In this dissertation, initially, a countermeasure technique is proposed for the detection of photograph attacks, which was our preliminary study that helps to gain insight into the topic of face spoofing. Then 3D mask attacks to face recognition systems are investigated. Compared to 2D spoofing attacks such as photograph and video, 3D mask attacks to face recognition systems is a considerably new subject. In this thesis, we evaluate how vulnerable the existing 3D and 2D face recognition systems are to spoofing mask attacks. The results of this study show that existing recognition systems are not robust against mask attacks and hence countermeasures have to be developed to mitigate their impact. Next, we analyze several characteristics of masks and real samples such as texture, reflectance and smoothness, and propose four countermeasure techniques for the detection of 3D mask attacks. All these countermeasures provide significant detection accuracies alone. However we observed that fusion of these countermeasures further improves the results. The second challenge in face recognition explored in this thesis is disguise variations. In the next part of this thesis, the disguise variations which are due to facial alterations, facial makeup and facial accessories (occlusions) are explored. Modifications on the facial surface can be achieved in countless different ways such as applying plastic surgery, prosthetic makeup and latex appliances. In our study regarding facial alterations, a simulated nose alteration database was prepared using FRGC v1.0, then the impact of nose alteration was evaluated on both 2D and 3D face recognition. Next, we propose a block based face recognition approach, which mitigates the impact of facial alterations on face recognition. Although this technique is applied for nose alterations in our study, it can be applied for any types of surgical operations on face. Facial makeup is another type of disguise variations. In order to analyze the impact of facial makeup on face recognition, we prepared a facial cosmetics database, which includes many samples annotated as non-makeup, slight makeup and heavy makeup. The impact of makeup on face recognition was analyzed in detail and we observed that especially makeup for eye region takes an important role in face recognition. Finally, in order to establish the connections between the Kinect sensor and face recognition, we present the first publicly available face database based on the Kinect sensor for face recognition. The database consists of different data modalities and multiple facial variations. We conduct benchmark evaluations on the proposed database using standard face recognition methods. The performance gain is demonstrated by integrating the depth data with the RGB data via score-level fusion. We also compare the Kinect images with the traditional high quality 3D scans (of the FRGC database which demonstrates the imperative needs of the proposed database for face recognition.