This dissertation studies soft biometrics traits, their applicability in different security and commercial scenarios, as well as related usability aspects. We place the emphasis on human facial soft biometric traits which constitute the set of physical, adhered or behavioral human characteristics that can partially differentiate, classify and identify humans. Such traits, which include characteristics like age, gender, skin and eye color, the presence of glasses, moustache or beard, inherit several advantages such as ease of acquisition, as well as a natural compatibility with how humans perceive their surroundings. Specifically, soft biometric traits are compatible with the human process of classifying and recalling our environment, a process which involves constructions of hierarchical structures of different refined traits. This thesis explores these traits, and their application in soft biometric systems (SBSs), and specifically focuses on how such systems can achieve different goals ...

Face recognition is an important technology in computer vision, which often acts as an essential component in biometrics systems, HCI systems, access control systems, multimedia indexing applications, etc. In recent years, identification of subjects in non-controlled scenarios has received large amount of attentions from the biometrics research community. The deployment of real-time and robust face recognition systems can significantly reinforce the safety and security in public places or/and private residences. However, variations due to expressions/illuminations/poses/occlusions can significantly deteriorate the performance of face recognition systems in non-controlled environments. Partial occlusion, which significantly changes the appearance of part of a face, cannot only cause large performance deterioration of face recognition, but also can cause severe security issues. In this thesis, we focus on the occlusion problem in automatic face recognition in noncontrolled environments. Toward this goal, we propose a framework that consists ...

This Thesis is focused on dealing with the variability factors in biometric recognition and applications of biometrics at a distance. In particular, this PhD Thesis explores the problem of variability factors assessment and how to deal with them by the incorporation of soft biometrics information in order to improve person recognition systems working at a distance. The proposed methods supported by experimental results show the benefits of adapting the system considering the variability of the sample at hand. Although being relatively young compared to other mature and long-used security technologies, biometrics have emerged in the last decade as a pushing alternative for applications where automatic recognition of people is needed. Certainly, biometrics are very attractive and useful for video surveillance systems at a distance, widely distributed in our lifes, and for the final user: forget about PINs and passwords, you ...

This thesis addresses the problem of drowsy driver detection using computer vision techniques applied to the human face. Specifically we explore the possibility of discriminating drowsy from alert video segments using facial expressions automatically extracted from video. Several approaches were previously proposed for the detection and prediction of drowsiness. There has recently been increasing interest in computer vision approaches as it is a potentially promising approach due to its non-invasive nature for detecting drowsiness. Previous studies with vision based approaches detect driver drowsiness primarily by making pre-assumptions about the relevant behavior, focusing on blink rate, eye closure, and yawning. Here we employ machine learning to explore, understand and exploit actual human behavior during drowsiness episodes. We have collected two datasets including facial and head movement measures. Head motion is collected through an accelerometer for the first dataset (UYAN-1) and an ...

Face recognition is one of the most challenging problems of computer vision and pattern recognition. The difficulty in face recognition arises mainly from facial appearance variations caused by factors, such as expression, illumination, partial face occlusion, and time gap between training and testing data capture. Moreover, the performance of face recognition algorithms heavily depends on prior facial feature localization step. That is, face images need to be aligned very well before they are fed into a face recognition algorithm, which requires precise facial feature localization. This thesis addresses on solving these two main problems -facial appearance variations due to changes in expression, illumination, occlusion, time gap, and imprecise face alignment due to mislocalized facial features- in order to accomplish its goal of building a generic face recognition algorithm that can function reliably under real-world conditions. The proposed face recognition algorithm ...

Human Activity Recognition HAR has been a hot research topic in the last decade due to its wide range of applications. Indeed, it has been the basis for implementa- tion of many computer vision applications, home security, video surveillance, and human-computer interaction. We intend by HAR, tools, and systems allowing to detect and recognize actions performed by individuals. With the considerable progress made in sensing technologies, HAR systems shifted from wearable and ambient-based to vision-based. This motivated the researchers to propose a large mass of vision-based solutions. From another perspective, HAR plays an impor- tant role in the health care sector and gets involved in the construction of fall detection systems and many smart home-related systems. Fall detection FD con- sists in identifying the occurrence of falls among other daily life activities. This is essential because falling is one of ...

Acutely-ill hospitalised patients are at risk of clinical deteriorations in health leading to adverse events such as cardiac arrests. Deteriorations are currently detected by manually measuring physiological parameters every 4-6 hours. Consequently, deteriorations can remain unrecognised between assessments, delaying clinical intervention. It may be possible to provide earlier detection of deteriorations by using wearable sensors for continuous physiological monitoring. Respiratory rate (RR) is not commonly monitored by wearable sensors, despite being a sensitive marker of deteriorations. This thesis presents investigations to identify an algorithm suitable for estimating RR from two signals commonly acquired by wearable sensors: the electrocardiogram (ECG) and photoplethysmogram (PPG). A suitable algorithm was then used to estimate RRs retrospectively from a physiological dataset acquired from acutely-ill patients to assess the potential utility of wearable sensors for detecting deteriorations. Existing RR algorithms were identi ed through a systematic ...

Advances in wearable electrocardiogram (ECG) monitoring devices have allowed for new cardiovascular applications to emerge beyond diagnostics, such as stress and fatigue detection, athletic performance assessment, sleep disorder characterization, mood recognition, activity surveillance, biometrics, and fitness tracking, to name a few. Such devices, however, are prone to artifacts, particularly due to movement, thus hampering heart rate and heart rate variability measurement and posing a serious threat to cardiac monitoring applications. To address these issues, this thesis proposes the use of a spectro-temporal signal representation called “modulation spectrum”, which is shown to accurately separate cardiac and noise components from the ECG signals, thus opening doors for noise-robust ECG signal processing tools and applications. First, an innovative ECG quality index based on the modulation spectral signal representation is proposed. The representation quantifies the rate-of-change of ECG spectral components, which are shown to ...

The number of cameras increases rapidly in squares, shopping centers, railway stations and airport halls. There are hundreds of cameras in the city center of Amsterdam. This is still modest compared to the tens of thousands of cameras in London, where citizens are expected to be filmed by more than three hundred cameras of over thirty separate Closed Circuit Television (CCTV) systems in a single day [84]. These CCTV systems include both publicly owned systems (railway stations, squares, airports) and privately owned systems (shops, banks, hotels). The main purpose of all these cameras is to detect, prevent and monitor crime and anti-social behaviour. Other goals of camera surveillance can be detection of unauthorized access, improvement of service, fire safety, etc. Since the terrorist attack on 9/11, detection and prevention of terrorist activities especially at high profiled locations such as airports, ...

In this doctoral dissertation, we principally explore the use of the temporal information available in video sequences for person and gender recognition; in particular, we focus on the analysis of head and facial motion, and their potential application as biometric identifiers. We also investigate how to exploit as much video information as possible for the automatic recognition; more precisely, we examine the possibility of integrating the head and mouth motion information with facial appearance into a multimodal biometric system, and we study the extraction of novel spatio-temporal facial features for recognition. We initially present a person recognition system that exploits the unconstrained head motion information, extracted by tracking a few facial landmarks in the image plane. In particular, we detail how each video sequence is firstly pre-processed by semiautomatically detecting the face, and then automatically tracking the facial landmarks over ...

Monitoring of vital signs is a key tool in medical diagnostics to asses the onset and the evolution of several diseases. Among fundamental vital parameters, such as the hearth rate, blood pressure and body temperature, the Respiratory Rate (RR) plays an important role. For this reason, respiration needs to be carefully monitored in order to detect potential signs or events indicating possible changes of health conditions. Monitoring of the respiration is generally carried out in hospital and clinical environments by the use of expensive devices with several sensors connected to the patient's body. A new research trend, in order to reduce healthcare service costs and make monitoring of vital signs more comfortable, is the development of low-cost systems which may allow remote and contactless monitoring; in such a context, an appealing method is to rely on video processing-based solutions. In ...

Automatic analysis of head gestures and facial expressions is a challenging research area and it has significant applications for intelligent human-computer interfaces. An important task is the automatic classification of non-verbal messages composed of facial signals where both facial expressions and head rotations are observed. This is a challenging task, because there is no definite grammar or code-book for mapping the non-verbal facial signals into a corresponding mental state. Furthermore, non-verbal facial signals and the observed emotions have dependency on personality, society, state of the mood and also the context in which they are displayed or observed. This thesis mainly addresses the three desired tasks for an effective visual information based automatic face and head gesture (FHG) analyzer. First we develop a fully automatic, robust and accurate 17-point facial landmark localizer based on local appearance information and structural information of ...

Cardiovascular diseases are a growing problem in today’s society. The World Health Organization (WHO) reported that these diseases make up about 30% of total global deaths and that heart diseases have no geographic, gender or socioeconomic boundaries. Therefore, detecting cardiac irregularities early-stage and a correct treatment are very important. However, this requires a good physiological understanding of the cardiovascular system. The heart is stimulated electrically by the brain via the autonomic nervous system, where sympathetic and vagal pathways are always interacting and modulating heart rate. Continuous monitoring of the heart activity is obtained by means of an ElectroCardioGram (ECG). Studying the fluctuations of heart beat intervals over time reveals a lot of information and is called heart rate variability (HRV) analysis. A reduction of HRV has been reported in several cardiological and noncardiological diseases. Moreover, HRV also has a prognostic ...

Current Human-Machine Interfaces (HMI) lack of “emotional intelligence”, i.e. they are not able to identify human emotional states and take this information into account to decide on the proper actions to execute. The goal of affective computing is to fill this lack by detecting emotional cues occurring during Human-Computer Interaction (HCI) and synthesizing emotional responses. In the last decades, most of the studies on emotion assessment have focused on the analysis of facial expressions and speech to determine the emotional state of a person. Physiological activity also includes emotional information that can be used for emotion assessment but has received less attention despite of its advantages (for instance it can be less easily faked than facial expressions). This thesis reports on the use of two types of physiological activities to assess emotions in the context of affective computing: the activity ...

Biometric recognition is the automated recognition of individuals based on their behavioral or biological characteristics. Beside forensic applications, this technology aims at replacing the outdated and attack prone, physical and knowledge-based, proofs of identity. Choosing one biometric characteristic is a tradeoff between universality, acceptability, and permanence, among other factors. Moreover, the accuracy cap of the chosen characteristic may limit the scalability and usability for some applications. The use of multiple biometric sources within a unified frame, i.e. multi-biometrics, aspires to tackle the limitations of single source biometrics and thus enables a wider implementation of the technology. This work aims at presenting application-driven advances in multi-biometrics by addressing different elements of the multi-biometric system work-flow. At first, practical oriented pre-fusion issues regarding missing data imputation and score normalization are discussed. This includes presenting a novel performance anchored score normalization technique that ...