Automated Face Recognition from Low-resolution Imagery

Recently, significant advances in the field of automated face recognition have been achieved using computer vision, machine learning, and deep learning methodologies. However, despite claims of super-human performance of face recognition algorithms on select key benchmark tasks, there remain several open problems that preclude the general replacement of human face recognition work with automated systems. State-of-the-art automated face recognition systems based on deep learning methods are able to achieve high accuracy when the face images they are tasked with recognizing subjects from are of sufficiently high quality. However, low image resolution remains one of the principal obstacles to face recognition systems, and their performance in the low-resolution regime is decidedly below human capabilities. In this PhD thesis, we present a systematic study of modern automated face recognition systems in the presence of image degradation in various forms. Based on our findings, we then propose a novel technique for improving the quality of low-resolution face images. Specifically, we present a novel deep learning model architecture for image super-resolution, and a novel training procedure for face hallucination that trains the model to super-resolve face images in a manner that preserves the information about the subject identity present in the low-resolution image. We validate the model by comparing its image reconstruction capability against several state-of-the-art models, as well as its performance on downstream semantic tasks including face recognition and face landmark localization. Next, we study the generalization capabilities of super-resolution-based face hallucination models, and find most of the models studied to be heavily biased towards the artificial image degradation process used to generate their training datasets. We notice that due to this bias, none of the face hallucination models considered are able to outperform an interpolation baseline on face recognition benchmarks with real-life low resolution images. To overcome this problem, we then develop a novel method for face recognition from low-resolution images that uses the results of multi-scale face hallucination models developed earlier. The proposed method is able to benefit from the high-resolution information added by the face hallucination models without suffering from the training set bias they exhibit, and systematically outperform the interpolation baseline and other state-of-the-art low-resolution face recognition models on the SCFace benchmark. Our proposed methods are trained on large face image datasets in a manner typical for deep learning models. However, the resulting trained models are useful for face recognition applications in an open-set regime, and do not need to be re-trained for novel subjects.