Bayesian Compressed Sensing using Alpha-Stable Distributions

During the last decades, information is being gathered and processed at an explosive rate. This fact gives rise to a very important issue, that is, how to effectively and precisely describe the information content of a given source signal or an ensemble of source signals, such that it can be stored, processed or transmitted by taking into consideration the limitations and capabilities of the several digital devices. One of the fundamental principles of signal processing for decades is the Nyquist-Shannon sampling theorem, which states that the minimum number of samples needed to reconstruct a signal without error is dictated by its bandwidth. However, there are many cases in our everyday life in which sampling at the Nyquist rate results in too many data and thus, demanding an increased processing power, as well as storage requirements. A mathematical theory that emerged recently presents the background for developing a novel sensing/sampling paradigm that goes against the common tenet in data acquisition. Compressed sensing (CS also known as compressive sensing, compressive sampling and sparse sampling, is a technique for acquiring and reconstructing a signal utilizing the prior knowledge that it is sparse or compressible, which provides a stricter sampling condition yielding a sub-Nyquist sampling criterion. Sparsity expresses the fact that the ?information rate? of a continuous-time signal may be much smaller than suggested by its bandwidth via the Nyquist?s theorem, or that a discrete-time signal depends on a number of degrees of freedom which is comparably much smaller than its (finite) length. Several deterministic and probabilistic approaches have been proposed over the last years confronting the problem of sparse signal reconstruction from distinct viewpoints. The majority of these methods relies on solving constrained-based optimization problems by employing several vector norms in the design of appropriate objective functions. Only recently the problem of CS reconstruction has been studied in a probabilistic (Bayesian) framework, resulting in several advantages when compared with the norm-based techniques. However, both of these classes of algorithms are based on a Gaussian assumption for the characterization of the statistics of the sparse signal. This thesis introduces the class of heavy-tailed distributions and particularly the family of alpha-Stable distributions as a suitable modeling tool for designing efficient CS reconstruction algorithms exploiting the sparsity of the received signal in an appropriate transform domain. More specifically, the first of the proposed methods exploits the prior knowledge for a sparse coefficient vector by modeling the statistics of its components using a Gaussian Scale Mixture (GSM). Then, the reconstruction of the sparse signal is reduced to the problem of estimating the parameters of the GSM model, which is in turn carried out by developing a Bayesian technique. Furthermore, there are applications, for instance in the case of a sensor network, where the acquisition process results in a set of multiple observations of the unknown sparse signal. For this purpose, we extend the previous method in order to take into account the fact that the set of multiple observations is characterized by a common sparsity structure with high probability, yielding an efficient CS method amenable to a distributed implementation. There are also real-world environments, such as in underwater acoustics or in telecommunications, where the signal and/or the noise present a highly impulsive behavior and thus, resulting in an even increased sparsity. The second method proposed in this thesis, which is also developed in a Bayesian framework, reconstructs signals corrupted by a highly impulsive noise component. This is done by modeling the statistical behavior of the sparse vector with a Cauchy distribution, which is also a member of the family of alpha-Stable distributions. The estimation of model parameters is performed by employing a well-known tree structure, which is a common approach in several Bayesian Learning tasks, such as the model selection as is the case here. Finally, a third method is proposed that generalizes the other two in the sense that it is developed for an arbitrary alpha-Stable distribution. The efficiency of a CS reconstruction method does not only depend on the inversion (decoding) method by itself, but also on a suitable encoding part (measurement matrix) that embeds all the significant information of the few large-amplitude transform coefficients into a measurement vector. So, it is shown first that such a measurement matrix is constructed exploiting the accuracy of an alpha-Stable distribution in modeling the highly impulsive (sparse) behavior of the sparse (transform) coefficients vector. The proposed CS algorithm for estimating a sparse vector proceeds by solving iteratively a constrained optimization problem using the duality theory and the method of subgradients. However, since the family of alpha-Stable distributions lacks finite variance, we introduce a modified Lagrangian function which takes into account the true non-Gaussian behavior as expressed by the so called Fractional Lower-Order Moments. In addition, it is shown that the objective function and the constraints are separable and thus, this algorithm is amenable to a distributed implementation from the nodes of a sensor network. We also illustrate the increased performance of the proposed CS algorithms based on heavy-tailed models when compared with the performance of recently introduced state-of-the-art CS reconstruction techniques, by applying them in a series of experiments for reconstructing simulated signals or for solving the problem of Direction-of-Arrival (DOA) estimation, as well as for recovering real image data from their corresponding transform coefficients.