Data-Driven Multimodal Signal Processing With Applications To EEG-fMRI Fusion

Most cognitive processes in the brain are reflected through several aspects simultaneously, allowing us to observe the same process from different biological phenomena. The diverse nature of these biological processes suggests that a better understanding of cerebral activity may be achieved through multimodal measurements. One of the possible multimodal brain recording settings is the combination of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI which is one of the main topics of this thesis. Two groups of EEG-fMRI integration approaches are possible. The first group, commonly called model-based techniques, are very popular due to the fact that the results from such analyses confirm or disprove a specific hypothesis. However, such hypotheses are not always available, requiring a more explorative approach to analyze the data. This exploration is possible with the second group of approaches, the so-called data-driven methods. The data-driven methods used in this work are based on blind source separation (BSS) and, more specifically, on independent component analysis (ICA) techniques. Besides the fusion of EEG and fMRI data, also the application of ICA to several types of single-channel and two-channels signals is studied. However, to be able to use ICA for these specific applications, modifications to the algorithm or to the way the data are used as input, are needed. Physiological signals are often measured from only one or a few electrodes, like, e.g., the electromyogram (EMG) measuring muscle activity. One of the limitations of ICA, however, is that it can extract independent components only when the number of sources embedded in the data is lower than, or equal to the number of recorded channels. In case of single-channel signals, this assumption is not satisfied. To solve this issue, in the first part of this thesis, we propose the use of empirical mode decomposition (EMD) to decompose a single-channel signal into a set of oscillatory modes, after which ICA is used to extract the underlying components. This method can also be extended to bivariate (or even multivariate) signals and is illustrated and validated in this thesis on data from baby cries, EEG and EMG data. The second part of this work is devoted to the application of ICA to EEG-fMRI fusion. EEG and fMRI data are structured together into one matrix and then jointly decomposed with ICA. This so-called JointICA algorithm [23] is first thoroughly validated on data from a simple visual detection task. The results of these analyses are compared to literature, and different aspects of the algorithm?s performance are discussed. JointICA is then used in a more complicated setting – the study of perceptual grouping. More specifically, the neural mechanisms of contour integration, i.e., the grouping of local edges into global contours are investigated. A modification of the algorithm is also proposed, allowing the comparison of different task conditions presented during the experimental paradigm. This allows to pinpoint detailed spatio-temporal differences and similarities across the presented conditions.