Audio-visual processing and content management techniques, for the study of (human) bioacoustics phenomena

The present doctoral thesis aims towards the development of new long-term, multi-channel, audio-visual processing techniques for the analysis of bioacoustics phenomena. The effort is focused on the study of the physiology of the gastrointestinal system, aiming at the support of medical research for the discovery of gastrointestinal motility patterns and the diagnosis of functional disorders. The term “processing” in this case is quite broad, incorporating the procedures of signal processing, content description, manipulation and analysis, that are applied to all the recorded bioacoustics signals, the auxiliary audio-visual surveillance information (for the monitoring of experiments and the subjects’ status and the extracted audio-video sequences describing the abdominal sound-field alterations. The thesis outline is as follows. The main objective of the thesis, which is the technological support of medical research, is presented in the first chapter. A quick problem definition is initially attempted, presenting the difficulties of gastrointestinal motility monitoring, as well as a thorough state of research of all the biomedical engineering approaches that have been utilized in this area, including bioacoustics methods. The technical specifications of the current medical research protocol are also presented, in combination with the proposed biomedical recording, processing and analysis system, aiming to resolve all the problems related with the prolonged monitoring period and the unsupervised nature of the experimental procedure. The second chapter focuses on the presentation of the data acquisition system, for both the bioacoustics signals and the audio-visual surveillance sequences. A quick theoretical introduction, including a state of research with the solutions employed in similar works, is initially deployed. It precedes the analysis of the current application demands and the design of a system capable of delivering the required functionalities, considering also easy future adaptation and collaboration with other psycho-physiology monitoring equipment. The modules that were tested and finally selected are then presented in a detailed manner, while conclusions and performance evaluation comments are also discussed. The third chapter concerns signal enhancement via audio de-noising techniques. This is a very important operation that affects both the medical auscultation of gastrointestinal sound signals as well as the various aspects of automated processing. In this context, the issues that cause the background noise generation and the difficulties emerged from the presence of additive broadband noise are discussed. De-noising theory is quickly presented, with emphasis on Wiener filtering and wavelet de-noising techniques that were combined in the development of a novel “Wavelet Domain Wiener Filtering” approach. The proposed methodology is then analyzed, in combination with the specific de-noising particularities and prospects, performance issues and computational demands. The algorithms, finally implemented, are tested in natural recordings and test signals, compared with existing methods, and rated via carefully setup qualitative and quantitative evaluation procedures. The fourth chapter presents long-term processing procedures, such as scanning techniques for the detection of significant bioacoustics events. Difficulties in prolonged recordings’ analysis and content manipulation are firstly discussed, in combination with the existed strategies in short-term and long-term abdominal sound analysis approaches. A new methodology is then described, combining both long-term summarization envelope signals, and short-term detected events. Windowed-based processing techniques and long-term de-nosing requirements are then analyzed. A new, multi-resolution scanning method, using wavelet energy comparison rules and fractal dimension formulas, is introduced, offering effective event detection, advanced indexing-based content manipulation, abstracting capabilities and topographic interpretation of the multi-channeled detected episodes. The performance of the new procedure is evaluated via qualitative and quantitative terms. Chapter five focuses on the process of determination of abdominal sound patterns. Many researchers consider that the lack of a standardised terminology and the absence of related classification dictionary, are some of the major obstacles towards the establishment of bowel-sound diagnostic methods. In the current thesis, a systematic experimental procedure was carried out to study potential classification schemes for abdominal sound patterns, with the help of the collaborative medical research team. The work was accomplished via the incorporation of all the existing medical knowledge and the standardisation of experimental observations, resulting to the definition of two classification dictionaries. Correspondingly, two pattern recognition expert systems were implemented, using neural networks and syntactic – hierarchic pattern classification approaches. Both tools were evaluated after their training and proved to exhibit remarkable performance and generalization capabilities. The importance of the pattern analysis task is twofold: it allows the isolation of the interfering noise patterns, and it facilitates more descriptive studying tools, such as the temporal and the spatio-temporal pattern distribution analysis approaches. The sixth chapter is focused on bioacoustics sound-field visualization, via sound source estimation algorithms and sound mapping tools. In this context, a general spatial-audio processing introduction is quickly presented, followed by a state of research analysis with the corresponding abdominal sound mapping techniques. A new sound-source location hybrid method is proposed, taking advantage of a three-axial accelerometer (analogously to the soundfield microphones) and the energy based localization numerical approaches. The implemented method is experimentally evaluated via synthetic abdominal propagation media and software simulation, overcoming the difficulties of older sound mapping approaches. Based on the sound localization results, various sound-field visualization tools are introduced in combination with surround sound reproduction (auscultation) techniques, to facilitate both short-term and long-term bowel-sound studies. The seventh chapter deals with the issue of audio-visual content manipulation and analysis. An MPEG-7 compatible ontology was setup for this purpose, aiming to provide advanced browsing and cross-search analysis capabilities, for both prolonged recording and isolated events. Thus, a quick MPEG-7 introduction is initially deployed, followed by a detailed presentation of the deigned ontology. The content management description schemes refer to all the experimental data, which are the multi-channeled abdominal sound recordings, the audio and video surveillance signals, and the bioacoustics’ sound-field audio-visual sequences. The proposed ontology is quite open, taking advantage of the MPEG-7 standard capabilities, and allowing easy future extension and adaptation to combined psycho-physiological monitoring protocols. In the eighth and last chapter the general conclusions on the total research contribution and the novelty aspects of the thesis, are presented. In general, we may distinguish two different areas that have been promoted from the current thesis results. First of all, the biomedical engineering field and the gastrointestinal medical sector are promoted, since a number of bio-signal processing methods and related analysis tools have been developed. Secondly, many of the proposed algorithms can be implemented or extended to be utilized to other audio-visual engineering applications. A complete list with the bibliographic reports utilized in this thesis is presented at the end of the corpus. The contribution of the work is focused on the following points: – Design and implementation of a new multi-channeled bioacoustic monitoring system that comprises the contact piezoelectric transducers, the tri-axial accelerometer, the additive sensors’ attaching and adaptation modules. The new system offers high spatial sensitivity, advanced topographic analysis capabilities and remarkable tolerance to ambient noise. – Development of an assistant audio-video surveillance system that allows for monitoring of the subjects and the experimental conditions, and facilitates automation procedures, via video motion detection techniques. – Implementation of novel wavelet-domain Wiener filtering de-noising techniques, for signal enhancement and removal of the unwanted background noise. The new algorithms feature remarkable performance to most of the known types of gastrointestinal sounds, in contrast to the preceding methods that were concentrated on explosive bowel sounds processing. Their reduced complexity and computational load makes them ideal for long-term processing demands. – Implementation of a new multi-resolution scanning procedure for event detection and abstraction of the prolonged recordings. The developed, wavelet-based long-term signal detection, segmentation and summarization method has been tested in natural recordings and test signals. It proved to be very efficient with compromised computational demands. – Systematical study of the gastrointestinal sound patterns, via interdisciplinary research collaboration, exhaustive screening, various clustering procedures and taking advantage of the existed knowledge. As a result, two hierarchical pattern classification dictionaries have been defined, in combination with the corresponding pattern recognition systems, offering new study methodologies for both short-term and long-term abdominal sound recordings. – Implementation of a new sound source localization algorithm, taking advantage of the tri-axial accelerometer components and the sound-energies of the peripheral piezoelectric transducers. Novel sound-field visualization and auralization tools have been developed facilitating the analysis process of the isolated bioacoustic events, and providing spatio-temporal summarization of the long-term gastrointestinal motility mechanisms. – Design and implementation of a MPEG-7 adapted ontology for data manipulation and analysis of all the involved audio-visual information. The proposed ontology introduces a novel description scheme, allowing for easy browsing, searching and retrieval. In this manner, it is helpful to all the medical approaches of bowel sound analysis, promising to deliver a standardized protocol for the study of gastrointestinal motility.