This work is concerned with two image-processing problems, image deconvolution with incomplete observations and data fusion of spectral images, and with some of the algorithms that are used to solve these and related problems. In image-deconvolution problems, the diagonalization of the blurring operator by means of the discrete Fourier transform usually yields very large speedups. When there are incomplete observations (e.g., in the case of unknown boundaries), standard deconvolution techniques normally involve non-diagonalizable operators, resulting in rather slow methods, or, otherwise, use inexact convolution models, resulting in the occurrence of artifacts in the enhanced images. We propose a new deconvolution framework for images with incomplete observations that allows one to work with diagonalizable convolution operators, and therefore is very fast. The framework is also an efficient, high-quality alternative to existing methods of dealing with the image boundaries, such as edge ...

Wireless sensor networks consist in a collection of battery-powered sensors able to gather, process and send data. They are typically used to monitor various phenomenons, in a plethora of fields, from environmental studies to smart logistics. Their wireless connectivity and relatively small size allow them to be deployed practically anywhere, even underwater or embedded in everyday clothing, and possibly capture data over a large area for extended periods of time. Their usefulness is therefore tied to their ability to work autonomously, with as little human intervention as possible. This functional requirement directly translates into two design constraints: (i) bandwidth and on-board compute must be used sparingly, in order to extend battery-life as much as possible, and (ii) the system must be resilient to node failures and changing environment. Due to their limited computing capabilities, data processing is usually performed by ...

As many machine learning and signal processing problems are fundamentally nonconvex and too expensive/difficult to be convexified, my research is focused on understanding the optimization landscapes of their fundamentally nonconvex formulations. After understanding their optimization landscapes, we can develop optimization algorithms to efficiently navigate these optimization landscapes and achieve the global optimality convergence. So, the main theme of this thesis would be optimization, with an emphasis on nonconvex optimization and algorithmic developments for these popular optimization problems. This thesis can be conceptually divided into four parts: Part 1: Convex Optimization. In the first part, we apply convex relaxations to several popular nonconvex problems in signal processing and machine learning (e.g. line spectral estimation problem and tensor decomposition problem) and prove that the solving the new convex relaxation problems is guaranteed to achieve the globally optimal solutions of their original nonconvex ...

The abundance of large and heterogeneous systems is rendering contemporary data more pervasive, intricate, and with a non-regular structure. With classical techniques facing troubles to deal with the irregular (non-Euclidean) domain where the signals are defined, a popular approach at the heart of graph signal processing (GSP) is to: (i) represent the underlying support via a graph and (ii) exploit the topology of this graph to process the signals at hand. In addition to the irregular structure of the signals, another critical limitation is that the observed data is prone to the presence of perturbations, which, in the context of GSP, may affect not only the observed signals but also the topology of the supporting graph. Ignoring the presence of perturbations, along with the couplings between the errors in the signal and the errors in their support, can drastically hinder ...

Typical acoustic scenes consist of multiple superimposed sources, where some of them represent desired signals, but often many of them are undesired sources, e.g., interferers or noise. Hence, source separation and extraction, i.e., the estimation of the desired source signals based on observed mixtures, is one of the central problems in audio signal processing. A promising class of approaches to address such problems is based on Independent Component Analysis (ICA), an unsupervised machine learning technique. These methods enjoyed a lot of attention from the research community due to the small number of assumptions that have to be made about the considered problem. Furthermore, the resulting generalization ability to unseen acoustic conditions, their mathematical rigor and the simplicity of resulting algorithms have been appreciated by many researchers working in audio signal processing. However, knowledge about the acoustic scenario is often available ...

The first part of this dissertation considers distributed learning problems over networked agents. The general objective of distributed adaptation and learning is the solution of global, stochastic optimization problems through localized interactions and without information about the statistical properties of the data. Regularization is a useful technique to encourage or enforce structural properties on the resulting solution, such as sparsity or constraints. A substantial number of regularizers are inherently non-smooth, while many cost functions are differentiable. We propose distributed and adaptive strategies that are able to minimize aggregate sums of objectives. In doing so, we exploit the structure of the individual objectives as sums of differentiable costs and non-differentiable regularizers. The resulting algorithms are adaptive in nature and able to continuously track drifts in the problem; their recursions, however, are subject to persistent perturbations arising from the stochastic nature of ...

The sparse coding is approximation/representation of signals with the minimum number of coefficients using an overcomplete set of elementary functions. This kind of approximations/ representations has found numerous applications in source separation, denoising, coding and compressed sensing. The adaptation of the sparse approximation framework to the coding problem of signals is investigated in this thesis. Open problems are the selection of appropriate models and their orders, coefficient quantization and sparse approximation method. Some of these questions are addressed in this thesis and novel methods developed. Because almost all recent communication and storage systems are digital, an easy method to compute quantized sparse approximations is introduced in the first part. The model selection problem is investigated next. The linear model can be adapted to better fit a given signal class. It can also be designed based on some a priori information ...

In this thesis we investigate the use of first-order convex optimization methods applied to problems in signal and image processing. First we make a general introduction to convex optimization, first-order methods and their iteration complexity. Then we look at different techniques, which can be used with first-order methods such as smoothing, Lagrange multipliers and proximal gradient methods. We continue by presenting different applications of convex optimization and notable convex formulations with an emphasis on inverse problems and sparse signal processing. We also describe the multiple-description problem. We finally present the contributions of the thesis. The remaining parts of the thesis consist of five research papers. The first paper addresses non-smooth first-order convex optimization and the trade-off between accuracy and smoothness of the approximating smooth function. The second and third papers concern discrete linear inverse problems and reliable numerical reconstruction software. ...

This thesis is based on nine papers, all concerned with parameter estimation. The thesis aims at solving problems related to real-world applications such as spectroscopy, DNA sequencing, and audio processing, using sparse modeling heuristics. For the problems considered in this thesis, one is not only concerned with finding the parameters in the signal model, but also to determine the number of signal components present in the measurements. In recent years, developments in sparse modeling have allowed for methods that jointly estimate the parameters in the model and the model order. Based on these achievements, the approach often taken in this thesis is as follows. First, a parametric model of the considered signal is derived, containing different parameters that capture the important characteristics of the signal. When the signal model has been determined, an optimization problem is formed aimed at finding ...

Radio resource allocation in communication networks is essential to achieve optimal performance and resource utilization. In modern interference networks the corresponding optimization problems are often nonconvex and their solution requires significant computational resources. Hence, practical systems usually use algorithms with no or only weak optimality guarantees for complexity reasons. Nevertheless, asserting the quality of these methods requires the knowledge of the globally optimal solution. State-of-the-art global optimization approaches mostly employ Tuy's monotonic optimization framework which has some major drawbacks, especially when dealing with fractional objectives or complicated feasible sets. In this thesis, two novel global optimization frameworks are developed. The first is based on the successive incumbent transcending (SIT) scheme to avoid numerical problems with complicated feasible sets. It inherently differentiates between convex and nonconvex variables, preserving the low computational complexity in the number of convex variables without the need ...

In multi-user (MU) downlink beamforming, a high spectral efficiency along with a low transmit power is achieved by separating multiple users in space rather than in time or frequency using spatially selective transmit beams. For streaming media applications, multi-group multicast (MGM) downlink beamforming is a promising approach to exploit the broadcasting property of the wireless medium to transmit the same information to a group of users. To limit inter-group interference, the individual streams intended for different multicast groups are spatially separated using MGM downlink beamforming. Spatially selective downlink beamforming requires the employment of an array of multiple antennas at the base station (BS). The hardware costs associated with the use of multiple antennas may be prohibitive in practice. A way to avoid the expensive employment of multiple antennas at the BS is to exploit user cooperation in wireless networks where ...

The major contribution of this thesis is on the problem of transmit beamforming to multiple cochannel multicast groups. Two viewpoints are considered: i) minimizing total transmission power while guaranteeing a prescribed minimum signal-to-interference-plus-noise ratio (SINR) at each receiver; and ii) a "fair" approach maximizing the overall minimum SINR under a total power budget. The core problem is a multicast generalization of the multiuser downlink beamforming problem; the difference is that each transmitted stream is directed to multiple receivers, each with its own channel. Such generalization is relevant and timely, e.g., in the context of the emerging WiMAX and UMTS-LTE wireless networks. The joint multicast beamforming problem is in general NP-hard, motivating the pursuit of computationally efficient quasi-optimal solutions. In chapter 1, it is shown that semidefinite relaxation coupled with suitable randomization / cochannel multicast power control yield computationally efficient high-quality ...

Damage to audio signals is in practice common, yet undesirable. Information loss can occur due to improper recording (low sample rate or dynamic range), transmission error (sample dropout), media damage, or because of noise. The removal of such disturbances is possible using inverse problems. Specifically, this work focuses on the situation where sections of an audio signal of length in the order of tens of milliseconds are completely lost, and the goal is to interpolate the missing samples based on the unimpaired context and a suitable signal model. The first part of the dissertation is devoted to convex and non-convex optimization methods, which are designed to find a solution to the interpolation problem based on the assumption of sparsity of the time-frequency spectrum. The general background and some algorithms are taken from the literature and adapted to the interpolation problem, ...

In this doctoral thesis several scale-free texture segmentation procedures based on two fractal attributes, the Hölder exponent, measuring the local regularity of a texture, and local variance, are proposed.A piecewise homogeneous fractal texture model is built, along with a synthesis procedure, providing images composed of the aggregation of fractal texture patches with known attributes and segmentation. This synthesis procedure is used to evaluate the proposed methods performance.A first method, based on the Total Variation regularization of a noisy estimate of local regularity, is illustrated and refined thanks to a post-processing step consisting in an iterative thresholding and resulting in a segmentation.After evidencing the limitations of this first approach, deux segmentation methods, with either "free" or "co-located" contours, are built, taking in account jointly the local regularity and the local variance.These two procedures are formulated as convex nonsmooth functional minimization problems.We ...

MULTIPLE-INPUT MULTIPLE-OUTPUT (MIMO) CHANNELS constitute a unified way of modeling a wide range of different physical communication channels, which can then be handled with a compact and elegant vector-matrix notation. The two paradigmatic examples are wireless multi-antenna channels and wireline Digital Subscriber Line (DSL) channels. Research in antenna arrays (also known as smart antennas) dates back to the 1960s. However, the use of multiples antennas at both the transmitter and the receiver, which can be naturally modeled as a MIMO channel, has been recently shown to offer a significant potential increase in capacity. DSL has gained popularity as a broadband access technology capable of reliably delivering high data rates over telephone subscriber lines. A DSL system can be modeled as a communication through a MIMO channel by considering all the copper twisted pairs within a binder as a whole rather ...