Facial Feature Extraction and Estimation of Gaze Direction in Human-Computer Interaction

In the modern age of information, there is a growing interest in improving interaction between humans and computers in an unremitting attempt to render it as seamless as the interaction between humans. In the core of this endeavor are the study of the human face and the focus of attention, determined by the eye gaze. The main objective of the current thesis is to develop accurate and reliable methods for extracting facial information, localizing the positions of the eye centers and performing tracking of the eye gaze. Usually such systems are grounded upon various assumptions regarding the topology of the features and the camera parameters or require dedicated hardware. In the regard of ubiquitous computing, all the methods developed in the scope of the current thesis use images and videos acquired using standard cameras under natural illumination, without the requirement of any special setup and without imposing constraints to the user. The methods developed for the facial feature extraction involve the detection of the lips, the eyebrows and the eyelids. The accurate localization of the eye centers is also extensively studied, comprising the cornerstone on which the study of eye gaze is based. Based on the methods of the previous chapters a system for estimating gaze direction is developed, studying also the different parameters that affect its performance and proposing the optimal setup. Finally, the gaze tracking system proposed is implemented in various practical applications, demonstrating its capabilities and appropriateness for real life applications. All the individual methods proposed have been extensively evaluated in challenging databases and compared to their counterparts found in the literature. The results presented in this thesis indicate the performance superiority of the proposed methods and encourage further research on the field.