A Robust Face Recognition Algorithm for Real-World Applications

Face recognition is one of the most challenging problems of computer vision and pattern recognition. The difficulty in face recognition arises mainly from facial appearance variations caused by factors, such as expression, illumination, partial face occlusion, and time gap between training and testing data capture. Moreover, the performance of face recognition algorithms heavily depends on prior facial feature localization step. That is, face images need to be aligned very well before they are fed into a face recognition algorithm, which requires precise facial feature localization. This thesis addresses on solving these two main problems -facial appearance variations due to changes in expression, illumination, occlusion, time gap, and imprecise face alignment due to mislocalized facial features- in order to accomplish its goal of building a generic face recognition algorithm that can function reliably under real-world conditions. The proposed face recognition algorithm is based on the representation of local facial regions using the discrete cosine transform (DCT). The local representation provides robustness against appearance variations in local regions caused by partial face occlusion or facial expression, whereas utilizing the frequency information provides robustness against changes in illumination. In addition, the algorithm bypasses the facial feature localization step and formulates face alignment as an optimization problem in the classification stage. Therefore, the system is free from the misalignment problem due to erroneous facial feature localization. The algorithm’s robustness against partial face occlusion, expression, illumination, time gap, and uncontrolled data capture conditions is first tested on five well-known benchmark face databases, namely on the AR, CMU PIE, FRGC, Yale B, and extended Yale B face databases. Extensive experiments have been conducted to analyze the effects of the algorithm’s parameters on the classification performance. Moreover, the algorithm’s robustness against image compression and registration errors is also assessed and it is compared with well-known generic face recognition algorithms. On all the experiments the algorithm attains very high correct recognition rates. It is found to be significantly superior to generic face recognition algorithms. It also outperforms, or performs as well as the algorithms that are designed specifically for just one type of factor that causes facial appearance variation, such as illumination. Experimental results show that, in the case of upper face occlusion caused by sunglasses, the main problem for low performance is not mainly because of missing eye region information but because of misalignment due to erroneous manual labeling of eye center positions. Since the algorithm is free from this problem, it also achieves very high correct recognition rates on this type of data. Several systems have been developed based on the proposed face recognition algorithm. In addition to the tests on the benchmark face databases, these systems are also evaluated on data collected under real-world conditions. One of the systems performs person identification in smart rooms and has been evaluated within the CLEAR evaluations. Other real-world applications, door monitoring, visitor interface, person identification in movies, have also been tested extensively. These evaluations show that the algorithm can work reliably under real-world conditions. The algorithm is also extended for a 3D face recognition scheme and found to perform successfully on the 3D data.