Most cognitive processes in the brain are reflected through several aspects simultaneously, allowing us to observe the same process from different biological phenomena. The diverse nature of these biological processes suggests that a better understanding of cerebral activity may be achieved through multimodal measurements. One of the possible multimodal brain recording settings is the combination of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), which is one of the main topics of this thesis. Two groups of EEG-fMRI integration approaches are possible. The first group, commonly called model-based techniques, are very popular due to the fact that the results from such analyses confirm or disprove a specific hypothesis. However, such hypotheses are not always available, requiring a more explorative approach to analyze the data. This exploration is possible with the second group of approaches, the so-called data-driven methods. The data-driven ...

A complex physical system like the human brain can only be comprehended by the use of a combination of various medical imaging techniques, each of which shed light on only a specific aspect of the neural processes that take place beneath the skull. Electroencephalography (EEG) and functional magnetic resonance (fMRI) are two such modalities, which enable the study of brain (dys)function. While the EEG is measured with a limited set of scalp electrodes which record rapid electrical changes resulting from neural activity, fMRI offers a superior spatial resolution at the expense of only picking up slow fluctuations of oxygen concentration that takes place near active brain cells. Hence, combining these very complementary modalities is an appealing, but complicated task due to their heterogeneous nature. In this thesis, we devise advanced signal processing techniques which integrate the multimodal data stemming from ...

Magnetic Resonance Imaging (MRI) constitutes a non-invasive medical imaging technique that allows the exploration of the inner anatomy, tissues, and physiological processes of the body. Among the different MRI applications, functional Magnetic Resonance Imaging (fMRI) has slowly become an essential tool for investigating the brain behavior and, nowadays, it plays a fundamental role in clinical and neurophysiological research. Due to its particular nature, specialized signal processing techniques are required in order to analyze the fMRI data properly. Among the various related techniques that have been developed over the years, the General Linear Model (GLM) is one of the most widely used approaches, and it usually appears as a default in many specialized software toolboxes for fMRI. On the other end, Blind Source Separation (BSS) methods constitute the most common alternative to GLM, especially when no prior information regarding the brain ...

Epilepsy is a neurological condition that manifests in epileptic seizures as a result of an abnormal, synchronous activity of a large group of neurons. Depending on the affected brain regions, seizures produce various severe clinical symptoms. Epilepsy cannot be cured and in many cases is not controlled by medication either. Surgical resection of the region responsible for generating the epileptic seizures might offer remedy for these patients. Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) measure the changes of brain activity in time over different locations of the brain. As such, they provide valuable information on the nature, the timing and the spatial origin of the epileptic activity. Unfortunately, both techniques record activity of different brain and artefact sources as well. Hence, EEG and fMRI signals are characterised by low signal to noise ratio. Data quality and the vast amount ...

The growing interest in neuroimaging technologies generates a massive amount of biomedical data that exhibit high dimensionality. Tensor-based analysis of brain imaging data has by now been recognized as an effective approach exploiting its inherent multi-way nature. In particular, the advantages of tensorial over matrix-based methods have previously been demonstrated in the context of functional magnetic resonance imaging (fMRI) source localization; the identification of the regions of the brain which are activated at specific time instances. However, such methods can also become ineffective in realistic challenging scenarios, involving, e.g., strong noise and/or significant overlap among the activated regions. Moreover, they commonly rely on the assumption of an underlying multilinear model generating the data. In the first part of this thesis, we aimed at investigating the possible gains from exploiting the 3-dimensional nature of the brain images, through a higher-order tensorization ...

Localizing the bioelectric phenomena originating from the cerebral cortex and evoked by auditory and somatosensory stimuli are clear objectives to both understand how the brain works and to recognize different pathologies. Diseases such as Parkinson's, Alzheimer's, schizophrenia and epilepsy are intensively studied to find a cure or accurate diagnosis. Epilepsy is considered the disease with major prevalence within disorders with neurological origin. The recurrent and sudden incidence of seizures can lead to dangerous and possibly life-threatening situations. Since disturbance of consciousness and sudden loss of motor control often occur without any warning, the ability to predict epileptic seizures would reduce patients' anxiety, thus considerably improving quality of life and safety. The common procedure for epilepsy seizure detection is based on brain activity monitorization via electroencephalogram (EEG) data. This process consumes a lot of time, especially in the case of long ...

Brain function requires a continuous flow of oxygen and glucose. An insufficient supply for a few minutes during the first period of life may have severe consequences or even result in death. This happens in one to six infants per 1000 live term births. Therefore, there is a high need for a method which can enable bedside brain monitoring to identify those neonates at risk and be able to start the treatment in time. The most important currently available technology to continuously monitor brain function is electroEncephaloGraphy (or EEG). Unfortunately, visual EEG analysis requires particular skills which are not always present round the clock in the Neonatal Intensive Care Unit (NICU). Even if those skills are available it is laborsome to manually analyse many hours of EEG. The lack of time and skill are the main reasons why EEG is ...

Work-related musculoskeletal disorders are a growing problem in todays society. These musculoskeletal disorders are caused by, amongst others, repetitive movements and mental stress. Stress is defined as the mismatch between a perceived demand and the perceived capacities to meet this demand. Although stress has a subjective origin, several physiological manifestations (e.g. cardiovascular and muscular) occur during periods of perceived stress. New insight and algorithms to extract information, related to stress are beneficial. Therefore, two series of stress experiments are executed in a laboratory environment, where subjects underwent different tasks inducing physical strain, mental stress and a combination of both. In this manuscript, new and modified algorithms for electromyography signals are presented that improve the individual analysis of electromyography signals. A first algorithm removes the interference of the electrical activity of the heart on singlechannel electromyography measurements. This interference signal is ...

Cochlear implants (CIs) aim to restore hearing in severely to profoundly deaf adults, children and infants. Electrically evoked auditory steady-state responses (EASSRs) are neural responses to continuous modulated pulse trains, and can be objectively detected at the modulation frequency in the electro-encephalogram (EEG). EASSRs provide a number of advantages over other objective measures, because frequency-specific stimuli are used, because targeted brain areas can be studied, depending on the chosen stimulation parameters, and because they can objectively be detected using statistical methods. EASSRs can potentially be used to determine appropriate stimulation levels during CI fitting, without behavioral input from the subjects. Furthermore, speech understanding in noise varies greatly between CI subjects. EASSRs lend themselves well to study the underlying causes of this variability, such as the integrity of the electrode-neuron interface or changes in the auditory cortex following deafness and following ...

A key issue in understanding how the brain functions is the ability to correlate functional information with anatomical localisation. Functional information can be provided by a variety of techniques like positron emission tomography (PET), functional MRI (fMRI), electroencephalography (EEG), magnetoencephalography (MEG) or transcranial magnetic stimulation (TMS). All these methods provide different, but complementary, information about the functional areas of the brain. PET and fMRI provide spatially accurate picture of brain regions involved in a given task. TMS permits to infer the contribution of the stimulated brain area to the task under investigation. EEG and MEG, which reflects brain activity directly, have temporal accuracy of the order of a millisecond. TMS, EEG and MEG are offset by their low spatial resolution. In this thesis, we propose two methods to improve the spatial accuracy of method based on TMS and EEG. The ...

The brain is the most important signal processing unit in the human body. It is responsible for receiving, processing and storing information. One of the possibilities to study brain functioning is by placing electrodes on the scalp and recording the synchronous neuronal activity of the brain. Such a recording measures a combination of active processes in the whole brain. Unfortunately, it is also contaminated by artifacts. By extracting the artifacts and removing them, cleaned recordings can be investigated. Furthermore, it is easier to look at specific brain activities, like an epileptic seizure, than at a combination. In this thesis, we present different mathematical techniques that can be used to extract individual contributing sources from the measured signals for this purpose. We focused on Canonical Correlation Analysis (CCA), Independent Component Analysis (ICA) and Canonical/ Parallel Factor Analysis (CPA). We show that ...

Identifying the target speaker in hearing aid applications is an essential ingredient to improve speech intelligibility. Although several speech enhancement algorithms are available to reduce background noise or to perform source separation in multi-speaker scenarios, their performance depends on correctly identifying the target speaker to be enhanced. Recent advances in electroencephalography (EEG) have shown that it is possible to identify the target speaker which the listener is attending to using single-trial EEG-based auditory attention decoding (AAD) methods. However, in realistic acoustic environments the AAD performance is influenced by undesired disturbances such as interfering speakers, noise and reverberation. In addition, it is important for real-world hearing aid applications to close the AAD loop by presenting on-line auditory feedback. This thesis deals with the problem of identifying and enhancing the target speaker in realistic acoustic environments based on decoding the auditory attention ...

Preterm birth is a major pediatric health problem that perturbs the genetically determined program of corticogenesis of the developing brain. As a consequence, prematurity has been strongly associated with adverse long-term neurodevelopmental outcome that may persist even into adulthood. Early characterization of the underlying neuronal mechanisms and early identification of infants at risk is of paramount importance since it allows better development of early therapeutic interventions aiming to prevent adverse outcomes through resilience. This dissertation aims to investigate the consequences of preterm birth on brain function and structure and their relation to adverse neurodevelopmental outcome, as well as to unveil the effect of an early music intervention on brain function. Research to date has mainly focused on the effect of early interventions on the long-term outcome but not on the effect of those interventions on brain function in preterm populations. ...

One desideratum in designing cognitive robots is autonomous learning of communication skills, just like humans. The primary step towards this goal is vocabulary acquisition. Being different from the training procedures of the state-of-the-art automatic speech recognition (ASR) systems, vocabulary acquisition cannot rely on prior knowledge of language in the same way. Like what infants do, the acquisition process should be data-driven with multi-level abstraction and coupled with multi-modal inputs. To avoid lengthy training efforts in a word-by-word interactive learning process, a clever learning agent should be able to acquire vocabularies from continuous speech automatically. The work presented in this thesis is entitled \emph{Constrained Non-negative Matrix Factorization for Vocabulary Acquisition from Continuous Speech}. Enlightened by the extensively studied techniques in ASR, we design computational models to discover and represent vocabularies from continuous speech with little prior knowledge of the language to ...

The ability to hear and process sounds is crucial. For adults, the inevitable ongoing aging process reduces the quality of the speech and sounds one perceives. If this effect is allowed to evolve too far, social isolation may occur. For infants, a disability in processing sounds results in an inappropriate development of speech, language, and cognitive abilities. To reduce the handicap of hearing loss in children, it is important to detect the hearing loss early and to provide effective rehabilitation. As a result, hearing of all newborns needs to be screened. If the outcome of the screening does not indicate normal hearing, more detailed hearing assessment is required. However, standard behavioral testing is not possible, so that assessment has to rely on objective physiological techniques that are not influenced by sleep or sedation. The last few decades, the use of ...