Bayesian methods for sparse and low-rank matrix problems

Many scientific and engineering problems require us to process measurements and data in order to extract information. Since we base decisions on information, it is important to design accurate and efficient processing algorithms. This is often done by modeling the signal of interest and the noise in the problem. One type of modeling is Compressed Sensing, where the signal has a sparse or low-rank representation. In this thesis we study different approaches to designing algorithms for sparse and low-rank problems. Greedy methods are fast methods for sparse problems which iteratively detects and estimates the non-zero components. By modeling the detection problem as an array processing problem and a Bayesian filtering problem, we improve the detection accuracy. Bayesian methods approximate the sparsity by probability distributions which are iteratively modified. We show one approach to making the Bayesian method the Relevance Vector ...

Sundin, Martin — Department of Signal Processing, Royal Institute of Technology KTH

Compressed sensing approaches to large-scale tensor decompositions

Today’s society is characterized by an abundance of data that is generated at an unprecedented velocity. However, much of this data is immediately thrown away by compression or information extraction. In a compressed sensing (CS) setting the inherent sparsity in many datasets is exploited by avoiding the acquisition of superfluous data in the first place. We combine this technique with tensors, or multiway arrays of numerical values, which are higher-order generalizations of vectors and matrices. As the number of entries scales exponentially in the order, tensor problems are often large-scale. We show that the combination of simple, low-rank tensor decompositions with CS effectively alleviates or even breaks the so-called curse of dimensionality. After discussing the larger data fusion optimization framework for coupled and constrained tensor decompositions, we investigate three categories of CS type algorithms to deal with large-scale problems. First, ...

Vervliet, Nico — KU Leuven

Epigraphical splitting of convex constraints. Application to image recovery, supervised classification, and image forgery detection.

In this thesis, we present a convex optimization approach to address three problems arising in multicomponent image recovery, supervised classification, and image forgery detection. The common thread among these problems is the presence of nonlinear convex constraints difficult to handle with state-of-the-art methods. Therefore, we present a novel splitting technique to simplify the management of such constraints. Relying on this approach, we also propose some contributions that are tailored to the aforementioned applications. The first part of the thesis presents the epigraphical splitting of nonlinear convex constraints. The principle is to decompose the sublevel set of a block-separable function into a collection of epigraphs. So doing, we reduce the complexity of optimization algorithms when the above constraint involves the sum of absolute values, distance functions to a convex set, Euclidean norms, infinity norms, or max functions. We demonstrate through numerical ...

Chierchia, Giovanni — Telecom ParisTech

Explicit and implicit tensor decomposition-based algorithms and applications

Various real-life data such as time series and multi-sensor recordings can be represented by vectors and matrices, which are one-way and two-way arrays of numerical values, respectively. Valuable information can be extracted from these measured data matrices by means of matrix factorizations in a broad range of applications within signal processing, data mining, and machine learning. While matrix-based methods are powerful and well-known tools for various applications, they are limited to single-mode variations, making them ill-suited to tackle multi-way data without loss of information. Higher-order tensors are a natural extension of vectors (first order) and matrices (second order), enabling us to represent multi-way arrays of numerical values, which have become ubiquitous in signal processing and data mining applications. By leveraging the powerful utitilies offered by tensor decompositions such as compression and uniqueness properties, we can extract more information from multi-way ...

Boussé, Martijn — KU Leuven

Efficient Globally Optimal Resource Allocation in Wireless Interference Networks

Radio resource allocation in communication networks is essential to achieve optimal performance and resource utilization. In modern interference networks the corresponding optimization problems are often nonconvex and their solution requires significant computational resources. Hence, practical systems usually use algorithms with no or only weak optimality guarantees for complexity reasons. Nevertheless, asserting the quality of these methods requires the knowledge of the globally optimal solution. State-of-the-art global optimization approaches mostly employ Tuy's monotonic optimization framework which has some major drawbacks, especially when dealing with fractional objectives or complicated feasible sets. In this thesis, two novel global optimization frameworks are developed. The first is based on the successive incumbent transcending (SIT) scheme to avoid numerical problems with complicated feasible sets. It inherently differentiates between convex and nonconvex variables, preserving the low computational complexity in the number of convex variables without the need ...

Matthiesen, Bho — Technische Universität Dresden, Dresden, Germany

Advanced Signal Processing Concepts for Multi-Dimensional Communication Systems

The widespread use of mobile internet and smart applications has led to an explosive growth in mobile data traffic. With the rise of smart homes, smart buildings, and smart cities, this demand is ever growing since future communication systems will require the integration of multiple networks serving diverse sectors, domains and applications, such as multimedia, virtual or augmented reality, machine-to-machine (M2M) communication / the Internet of things (IoT), automotive applications, and many more. Therefore, in the future, the communication systems will not only be required to provide Gbps wireless connectivity but also fulfill other requirements such as low latency and massive machine type connectivity while ensuring the quality of service. Without significant technological advances to increase the system capacity, the existing telecommunications infrastructure will be unable to support these multi-dimensional requirements. This poses an important demand for suitable waveforms with ...

Cheema, Sher Ali — Technische Universität Ilmenau

Sparse Modeling Heuristics for Parameter Estimation - Applications in Statistical Signal Processing

This thesis examines sparse statistical modeling on a range of applications in audio modeling, audio localizations, DNA sequencing, and spectroscopy. In the examined cases, the resulting estimation problems are computationally cumbersome, both as one often suffers from a lack of model order knowledge for this form of problems, but also due to the high dimensionality of the parameter spaces, which typically also yield optimization problems with numerous local minima. In this thesis, these problems are treated using sparse modeling heuristics, with the resulting criteria being solved using convex relaxations, inspired from disciplined convex programming ideas, to maintain tractability. The contributions to audio modeling and estimation focus on the estimation of the fundamental frequency of harmonically related sinusoidal signals, which is commonly used model for, e.g., voiced speech or tonal audio. We examine both the problems of estimating multiple audio sources ...

Adalbjörnsson, Stefan Ingi — Lund University

Density-based shape descriptors and similarity learning for 3D object retrieval

Next generation search engines will enable query formulations, other than text, relying on visual information encoded in terms of images and shapes. The 3D search technology, in particular, targets specialized application domains ranging from computer aided-design and manufacturing to cultural heritage archival and presentation. Content-based retrieval research aims at developing search engines that would allow users to perform a query by similarity of content. This thesis deals with two fundamentals problems in content-based 3D object retrieval: (1) How to describe a 3D shape to obtain a reliable representative for the subsequent task of similarity search? (2) How to supervise the search process to learn inter-shape similarities for more effective and semantic retrieval? Concerning the first problem, we develop a novel 3D shape description scheme based on probability density of multivariate local surface features. We constructively obtain local characterizations of 3D ...

Akgul, Ceyhun Burak — Bogazici University and Telecom ParisTech

Functional Neuroimaging Data Characterisation Via Tensor Representations

The growing interest in neuroimaging technologies generates a massive amount of biomedical data that exhibit high dimensionality. Tensor-based analysis of brain imaging data has by now been recognized as an effective approach exploiting its inherent multi-way nature. In particular, the advantages of tensorial over matrix-based methods have previously been demonstrated in the context of functional magnetic resonance imaging (fMRI) source localization; the identification of the regions of the brain which are activated at specific time instances. However, such methods can also become ineffective in realistic challenging scenarios, involving, e.g., strong noise and/or significant overlap among the activated regions. Moreover, they commonly rely on the assumption of an underlying multilinear model generating the data. In the first part of this thesis, we aimed at investigating the possible gains from exploiting the 3-dimensional nature of the brain images, through a higher-order tensorization ...

Christos Chatzichristos — National and Kapodistrian University of Athens

A Unified Framework for Communications through MIMO Channels

MULTIPLE-INPUT MULTIPLE-OUTPUT (MIMO) CHANNELS constitute a unified way of modeling a wide range of different physical communication channels, which can then be handled with a compact and elegant vector-matrix notation. The two paradigmatic examples are wireless multi-antenna channels and wireline Digital Subscriber Line (DSL) channels. Research in antenna arrays (also known as smart antennas) dates back to the 1960s. However, the use of multiples antennas at both the transmitter and the receiver, which can be naturally modeled as a MIMO channel, has been recently shown to offer a significant potential increase in capacity. DSL has gained popularity as a broadband access technology capable of reliably delivering high data rates over telephone subscriber lines. A DSL system can be modeled as a communication through a MIMO channel by considering all the copper twisted pairs within a binder as a whole rather ...

Palomar, Daniel Perez — Technical University of Catalonia (UPC)

Geometric Approach to Statistical Learning Theory through Support Vector Machines (SVM) with Application to Medical Diagnosis

This thesis deals with problems of Pattern Recognition in the framework of Machine Learning (ML) and, specifically, Statistical Learning Theory (SLT), using Support Vector Machines (SVMs). The focus of this work is on the geometric interpretation of SVMs, which is accomplished through the notion of Reduced Convex Hulls (RCHs), and its impact on the derivation of new, efficient algorithms for the solution of the general SVM optimization task. The contributions of this work is the extension of the mathematical framework of RCHs, the derivation of novel geometric algorithms for SVMs and, finally, the application of the SVM algorithms to the field of Medical Image Analysis and Diagnosis (Mammography). Geometric SVM Framework's extensions: The geometric interpretation of SVMs is based on the notion of Reduced Convex Hulls. Although the geometric approach to SVMs is very intuitive, its usefulness was restricted by ...

Mavroforakis, Michael — University of Athens

Efficient matrices for signal processing and machine learning. (Matrices efficientes pour le traitement du signal et l'apprentissage automatique.)

Matrices, as natural representation of linear mappings in finite dimension, play a crucial role in signal processing and machine learning. Multiplying a vector by a full rank matrix a priori costs of the order of the number of non-zero entries in the matrix, in terms of arithmetic operations. However, matrices exist that can be applied much faster, this property being crucial to the success of certain linear transformations, such as the Fourier transform or the wavelet transform. What is the property that allows these matrices to be applied rapidly ? Is it easy to verify ? Can weapproximate matrices with ones having this property ? Can we estimate matrices having this property ? This thesis investigates these questions, exploring applications such as learning dictionaries with efficient implementations, accelerating the resolution of inverse problems or Fast Fourier Transform on graphs.

Le Magoarou, Luc — INRIA, Technicolor

Learning from structured EEG and fMRI data supporting the diagnosis of epilepsy

Epilepsy is a neurological condition that manifests in epileptic seizures as a result of an abnormal, synchronous activity of a large group of neurons. Depending on the affected brain regions, seizures produce various severe clinical symptoms. Epilepsy cannot be cured and in many cases is not controlled by medication either. Surgical resection of the region responsible for generating the epileptic seizures might offer remedy for these patients. Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) measure the changes of brain activity in time over different locations of the brain. As such, they provide valuable information on the nature, the timing and the spatial origin of the epileptic activity. Unfortunately, both techniques record activity of different brain and artefact sources as well. Hence, EEG and fMRI signals are characterised by low signal to noise ratio. Data quality and the vast amount ...

Hunyadi, Borbála — KU Leuven

Contributions to signal analysis and processing using compressed sensing techniques

Chapter 2 contains a short introduction to the fundamentals of compressed sensing theory, which is the larger context of this thesis. We start with introducing the key concepts of sparsity and sparse representations of signals. We discuss the central problem of compressed sensing, i.e. how to adequately recover sparse signals from a small number of measurements, as well as the multiple formulations of the reconstruction problem. A large part of the chapter is devoted to some of the most important conditions necessary and/or sufficient to guarantee accurate recovery. The aim is to introduce the reader to the basic results, without the burden of detailed proofs. In addition, we also present a few of the popular reconstruction and optimization algorithms that we use throughout the thesis. Chapter 3 presents an alternative sparsity model known as analysis sparsity, that offers similar recovery ...

Cleju, Nicolae — "Gheorghe Asachi" Technical University of Iasi

Parameter Estimation -in sparsity we trust

This thesis is based on nine papers, all concerned with parameter estimation. The thesis aims at solving problems related to real-world applications such as spectroscopy, DNA sequencing, and audio processing, using sparse modeling heuristics. For the problems considered in this thesis, one is not only concerned with finding the parameters in the signal model, but also to determine the number of signal components present in the measurements. In recent years, developments in sparse modeling have allowed for methods that jointly estimate the parameters in the model and the model order. Based on these achievements, the approach often taken in this thesis is as follows. First, a parametric model of the considered signal is derived, containing different parameters that capture the important characteristics of the signal. When the signal model has been determined, an optimization problem is formed aimed at finding ...

Swärd, Johan — Lund University

The current layout is optimized for mobile phones. Page previews, thumbnails, and full abstracts will remain hidden until the browser window grows in width.

The current layout is optimized for tablet devices. Page previews and some thumbnails will remain hidden until the browser window grows in width.