Many scientific and engineering problems require us to process measurements and data in order to extract information. Since we base decisions on information, it is important to design accurate and efficient processing algorithms. This is often done by modeling the signal of interest and the noise in the problem. One type of modeling is Compressed Sensing, where the signal has a sparse or low-rank representation. In this thesis we study different approaches to designing algorithms for sparse and low-rank problems. Greedy methods are fast methods for sparse problems which iteratively detects and estimates the non-zero components. By modeling the detection problem as an array processing problem and a Bayesian filtering problem, we improve the detection accuracy. Bayesian methods approximate the sparsity by probability distributions which are iteratively modified. We show one approach to making the Bayesian method the Relevance Vector ...

This thesis develops new mathematical theory and presents novel recovery algorithms for discrete linear dynamical systems (LDS) with sparsity constraints on either control inputs or initial state. The recovery problems in this framework manifest as the problem of reconstructing one or more sparse signals from a set of noisy underdetermined linear measurements. The goal of our work is to design algorithms for sparse signal recovery which can exploit the underlying structure in the measurement matrix and the unknown sparse vectors, and to analyze the impact of these structures on the efficacy of the recovery. We answer three fundamental and interconnected questions on sparse signal recovery problems that arise in the context of LDS. First, what are necessary and sufficient conditions for the existence of a sparse solution? Second, given that a sparse solution exists, what are good low-complexity algorithms that ...

This work aims at exploiting compressive sensing paradigms in order to reduce the cost of statistical learning tasks. We first provide a reminder of compressive sensing bases and describe some statistical analysis tasks using similar ideas. Then we describe a framework to perform parameter estimation on probabilistic mixture models in a case where training data is compressed to a fixed-size representation called a sketch. We formulate the estimation as a generalized inverse problem for which we propose a greedy algorithm. We experiment this framework and algorithm on an isotropic Gaussian mixture model. This proof of concept suggests the existence of theoretical recovery guarantees for sparse objects beyond the usual vector and matrix cases. We therefore study the generalization of linear inverse problems stability results on general signal models encompassing the standard cases and the sparse mixtures of probability distributions. We ...

Compressed sensing (CS) is a recently introduced signal acquisition framework that goes against the traditional Nyquist sampling paradigm. CS demonstrates that a sparse, or compressible, signal can be acquired using a low rate acquisition process. Since noise is always present in practical data acquisition systems, sensing and reconstruction methods are developed assuming a Gaussian (light-tailed) model for the corrupting noise. However, when the underlying signal and/or the measurements are corrupted by impulsive noise, commonly employed linear sampling operators, coupled with Gaussian-derived reconstruction algorithms, fail to recover a close approximation of the signal. This dissertation develops robust sampling and reconstruction methods for sparse signals in the presence of impulsive noise. To achieve this objective, we make use of robust statistics theory to develop appropriate methods addressing the problem of impulsive noise in CS systems. We develop a generalized Cauchy distribution (GCD) ...

In lossless coding, compresssion and decompression of source data result in the exact recovery of the individual elements of the original source data. Lossless image / video coding is necessary in applications where no loss of pixel values is tolerable. Examples are medical imaging, remote sensing, in image/video archives and studio applications where tandem- and trans-coding are used in editing, which can lead to accumulating errors. Nearly-lossless coding is used in applications where a small error, defined as a maximum error or as a root mean square (rms) error, is tolerable. In lossless embedded coding, a losslessly coded bit stream can be decoded at any bit rate lower than the lossless bit rate. In this thesis, research on embedded lossless video coding based on a motion compensated framework, similar to that of MPEG-2, is presented. Transforms that map integers into ...

This dissertation details three approaches for direction-of-arrival (DOA) estimation or beamforming in array signal processing from the perspective of sparsity. In the first part of this dissertation, we consider sparse array beamformer design based on the alternating direction method of multipliers (ADMM); in the second part of this dissertation, the problem of joint DOA estimation and distorted sensor detection is investigated; and off-grid DOA estimation is studied in the last part of this dissertation. In the first part of this thesis, we devise a sparse array design algorithm for adaptive beamforming. Our strategy is based on finding a sparse beamformer weight to maximize the output signal-to-interference-plus-noise ratio (SINR). The proposed method utilizes ADMM, and admits closed-form solutions at each ADMM iteration. The algorithm convergence properties are analyzed by showing the monotonicity and boundedness of the augmented Lagrangian function. In addition, ...

The sparse coding is approximation/representation of signals with the minimum number of coefficients using an overcomplete set of elementary functions. This kind of approximations/ representations has found numerous applications in source separation, denoising, coding and compressed sensing. The adaptation of the sparse approximation framework to the coding problem of signals is investigated in this thesis. Open problems are the selection of appropriate models and their orders, coefficient quantization and sparse approximation method. Some of these questions are addressed in this thesis and novel methods developed. Because almost all recent communication and storage systems are digital, an easy method to compute quantized sparse approximations is introduced in the first part. The model selection problem is investigated next. The linear model can be adapted to better fit a given signal class. It can also be designed based on some a priori information ...

Ill-posed inverse problems appear in many signal and image processing applications, such as deblurring, super-resolution and compressed sensing. The common approach to address them is to design a specific algorithm, or recently, a specific deep neural network, for each problem. Both signal processing and machine learning tactics have drawbacks: traditional reconstruction strategies exhibit limited performance for complex signals, such as natural images, due to the hardness of their mathematical modeling; while modern works that circumvent signal modeling by training deep convolutional neural networks (CNNs) suffer from a huge performance drop when the observation model used in training is inexact. In this work, we develop and analyze reconstruction algorithms that are not restricted to a specific signal model and are able to handle different observation models. Our main contributions include: (a) We generalize the popular sparsity-based CoSaMP algorithm to any signal ...

Multi-antenna processing is widely adopted as one of the key enabling technologies for current and future cellular networks. Particularly, multiuser downlink beamforming (also known as space-division multiple access), in which multiple users are simultaneously served with spatial transmit beams in the same time and frequency resource, achieves high spectral efficiency with reduced energy consumption. To harvest the potential of multiuser downlink beamforming in practical systems, optimal beamformer design shall be carried out jointly with network resource allocation. Due to the specifications of cellular standards and/or implementation constraints, resource allocation in practice naturally necessitates discrete decision makings, e.g., base station (BS) association, user scheduling and admission control, adaptive modulation and coding, and codebook-based beamforming (precoding). This dissertation focuses on the joint optimization of multiuser downlink beamforming and discrete resource allocation in modern cellular networks. The problems studied in this thesis involve ...

Chapter 2 contains a short introduction to the fundamentals of compressed sensing theory, which is the larger context of this thesis. We start with introducing the key concepts of sparsity and sparse representations of signals. We discuss the central problem of compressed sensing, i.e. how to adequately recover sparse signals from a small number of measurements, as well as the multiple formulations of the reconstruction problem. A large part of the chapter is devoted to some of the most important conditions necessary and/or sufficient to guarantee accurate recovery. The aim is to introduce the reader to the basic results, without the burden of detailed proofs. In addition, we also present a few of the popular reconstruction and optimization algorithms that we use throughout the thesis. Chapter 3 presents an alternative sparsity model known as analysis sparsity, that offers similar recovery ...

This dissertation is the combination of three manuscripts –either published in or submitted to journals– on compressive sensing of propeller noise for detection, identification and localization of water crafts. Propeller noise, as a result of rotating blades, is broadband and radiates through water dominating underwater acoustic noise spectrum especially when cavitation develops. Propeller cavitation yields cyclostationary noise which can be modeled by amplitude modulation, i.e., the envelope-carrier product. The envelope consists of the so-called propeller tonals representing propeller characteristics which is used to identify water crafts whereas the carrier is a stationary broadband process. Sampling for propeller noise processing yields large data sizes due to Nyquist rate and multiple sensor deployment. A compressive sensing scheme is proposed for efficient sampling of second-order cyclostationary propeller noise since the spectral correlation function of the amplitude modulation model is sparse as shown in ...

Sparsity-based estimation techniques deal with the problem of retrieving a data vector from an undercomplete set of linear observations, when the data vector is known to have few nonzero elements with unknown positions. It is also known as the atomic decomposition problem, and has been carefully studied in the field of compressed sensing. Recent findings have led to a method called basis pursuit, also known as Least Absolute Shrinkage and Selection Operator (LASSO), as a numerically reliable sparsity-based approach. Although the atomic decomposition problem is generally NP-hard, it has been shown that basis pursuit may provide exact solutions under certain assumptions. This has led to an extensive study of signals with sparse representation in different domains, providing a new general insight into signal processing. This thesis further investigates the role of sparsity-based techniques, especially basis pursuit, for solving parameter estimation ...

This doctorate thesis focuses on sparse regression, a statistical modeling tool for selecting valuable predictors in underdetermined linear models. By imposing different constraints on the structure of the variable vector in the regression problem, one obtains estimates which have sparse supports, i.e., where only a few of the elements in the response variable have non-zero values. The thesis collects six papers which, to a varying extent, deals with the applications, implementations, modifications, translations, and other analysis of such problems. Sparse regression is often used to approximate additive models with intricate, non-linear, non-smooth or otherwise problematic functions, by creating an underdetermined model consisting of candidate values for these functions, and linear response variables which selects among the candidates. Sparse regression is therefore a widely used tool in applications such as, e.g., image processing, audio processing, seismological and biomedical modeling, but is ...

In multi-user (MU) downlink beamforming, a high spectral efficiency along with a low transmit power is achieved by separating multiple users in space rather than in time or frequency using spatially selective transmit beams. For streaming media applications, multi-group multicast (MGM) downlink beamforming is a promising approach to exploit the broadcasting property of the wireless medium to transmit the same information to a group of users. To limit inter-group interference, the individual streams intended for different multicast groups are spatially separated using MGM downlink beamforming. Spatially selective downlink beamforming requires the employment of an array of multiple antennas at the base station (BS). The hardware costs associated with the use of multiple antennas may be prohibitive in practice. A way to avoid the expensive employment of multiple antennas at the BS is to exploit user cooperation in wireless networks where ...

Compressed Sensing (CS) is a widely used technique for efficient signal acquisition, in which a very small number of (possibly noisy) linear measurements of an unknown signal vector are taken via multiplication with a designed ‘sensing matrix’ in an application-specific manner, and later recovered by exploiting the sparsity of the signal vector in some known orthonormal basis and some special properties of the sensing matrix which allow for such recovery. We study three new applications of CS, each of which poses a unique challenge in a different aspect of it, and propose novel techniques to solve them, advancing the field of CS. Each application involves a unique combination of realistic assumptions on the measurement noise model and the signal, and a unique set of algorithmic challenges. We frame Pooled RT-PCR Testing for COVID-19 – wherein RT-PCR (Reverse Transcription Polymerase Chain ...