## Distributed Stochastic Optimization in Non-Differentiable and Non-Convex Environments (2019)

Group-Sparse Regression - With Applications in Spectral Analysis and Audio Signal Processing

This doctorate thesis focuses on sparse regression, a statistical modeling tool for selecting valuable predictors in underdetermined linear models. By imposing different constraints on the structure of the variable vector in the regression problem, one obtains estimates which have sparse supports, i.e., where only a few of the elements in the response variable have non-zero values. The thesis collects six papers which, to a varying extent, deals with the applications, implementations, modifications, translations, and other analysis of such problems. Sparse regression is often used to approximate additive models with intricate, non-linear, non-smooth or otherwise problematic functions, by creating an underdetermined model consisting of candidate values for these functions, and linear response variables which selects among the candidates. Sparse regression is therefore a widely used tool in applications such as, e.g., image processing, audio processing, seismological and biomedical modeling, but is ...

Kronvall, Ted — Lund University

Algorithms for Energy-Efficient Adaptive Wireless Sensor Networks

In this thesis we focus on the development of energy-efficient adaptive algorithms for Wireless Sensor Networks. Its contributions can be arranged in two main lines. Firstly, we focus on the efficient management of energy resources in WSNs equipped with finite-size batteries and energy-harvesting devices. To that end, we propose a censoring scheme by which the nodes are able to decide if a message transmission is worthy or not given their energetic condition. In order to do so, we model the system using a Markov Decision Process and use this model to derive optimal policies. Later, these policies are analyzed in simplified scenarios in order to get insights of their features. Finally, using Stochastic Approximation, we develop low-complexity censoring algorithms that approximate the optimal policy, with less computational complexity and faster convergence speed than other approaches such as Q-learning. Secondly, we ...

Fernandez-Bes, Jesus — Universidad Carlos III de Madrid

Learning Transferable Knowledge through Embedding Spaces

The unprecedented processing demand, posed by the explosion of big data, challenges researchers to design efficient and adaptive machine learning algorithms that do not require persistent retraining and avoid learning redundant information. Inspired from learning techniques of intelligent biological agents, identifying transferable knowledge across learning problems has been a significant research focus to improve machine learning algorithms. In this thesis, we address the challenges of knowledge transfer through embedding spaces that capture and store hierarchical knowledge. In the first part of the thesis, we focus on the problem of cross-domain knowledge transfer. We first address zero-shot image classification, where the goal is to identify images from unseen classes using semantic descriptions of these classes. We train two coupled dictionaries which align visual and semantic domains via an intermediate embedding space. We then extend this idea by training deep networks that ...

Mohammad Rostami — University of Pennsylvania

Stochastic Schemes for Dynamic Network Resource Allocation

Wireless networks and power distribution grids are experiencing increasing demands on their efficiency and reliability. Judicious methods for allocating scarce resources such as power and bandwidth are of paramount importance. As a result, nonlinear optimization and signal processing tools have been incorporated into the design of contemporary networks. This thesis develops schemes for efficient resource allocation (RA) in such dynamic networks, with an emphasis in stochasticity, which is accounted for in the problem formulation as well as in the algorithms and schemes to solve those problems. Stochastic optimization and decomposition techniques are investigated to develop low-complexity algorithms with specific applications in cross-layer design of wireless communications, cognitive radio (CR) networks and smart power distribution systems. The costs and constraints on the availability of network resources, together with diverse quality of service (QoS) requirements, render network design, management, and operation challenging ...

Lopez Ramos, Luis Miguel — King Juan Carlos University

This dissertation deals with the distributed processing techniques for parameter estimation and efficient data-gathering in wireless communication and sensor networks. The estimation problem consists in inferring a set of parameters from temporal and spatial noisy observations collected by different nodes that monitor an area or field. The objective is to derive an estimate that is as accurate as the one that would be obtained if each node had access to the information across the entire network. With the aim of enabling an energy aware and low-complexity distributed implementation of the estimation task, several useful optimization techniques that generally yield linear estimators were derived in the literature. Up to now, most of the works considered that the nodes are interested in estimating the same vector of global parameters. This scenario can be viewed as a special case of a more general ...

Bogdanovic, Nikola — University of Patras

GRAPH-TIME SIGNAL PROCESSING: FILTERING AND SAMPLING STRATEGIES

The necessity to process signals living in non-Euclidean domains, such as signals de- fined on the top of a graph, has led to the extension of signal processing techniques to the graph setting. Among different approaches, graph signal processing distinguishes it- self by providing a Fourier analysis of these signals. Analogously to the Fourier transform for time and image signals, the graph Fourier transform decomposes the graph signals in terms of the harmonics provided by the underlying topology. For instance, a graph signal characterized by a slow variation between adjacent nodes has a low frequency content. Along with the graph Fourier transform, graph filters are the key tool to alter the graph frequency content of a graph signal. This thesis focuses on graph filters that are performed distributively in the node domain–that is, each node needs to exchange in- formation ...

Elvin Isufi — Delft University of Technology

Resistivity distribution estimation, widely known as Electrical Impedance Tomography (EIT), is a non linear ill-posed inverse problem. However, the partial derivative equation ruling this experiment yields no analytical solution for arbitrary conductivity distribution. Thus, solving the forward problem requires an approximation. The Finite Element Method (FEM) provides us with a computationally cheap forward model which preserves the non linear image-data relation and also reveals sufficiently accurate for the inversion. Within the Bayesian approach, Markovian priors on the log-conductivity distribution are introduced for regularization. The neighborhood system is directly derived from the FEM triangular mesh structure. We first propose a maximum a posteriori (MAP) estimation with a Huber-Markov prior which favours smooth distributions while preserving locally discontinuous features. The resulting criterion is minimized with the pseudo-conjugate gradient method. Simulation results reveal significant improvements in terms of robustness to noise, computation rapidity ...

Martin, Thierry — Laboratoire des signaux et systèmes

Convex and Nonconvex Optimization Geometries

As many machine learning and signal processing problems are fundamentally nonconvex and too expensive/difficult to be convexified, my research is focused on understanding the optimization landscapes of their fundamentally nonconvex formulations. After understanding their optimization landscapes, we can develop optimization algorithms to efficiently navigate these optimization landscapes and achieve the global optimality convergence. So, the main theme of this thesis would be optimization, with an emphasis on nonconvex optimization and algorithmic developments for these popular optimization problems. This thesis can be conceptually divided into four parts: Part 1: Convex Optimization. In the first part, we apply convex relaxations to several popular nonconvex problems in signal processing and machine learning (e.g. line spectral estimation problem and tensor decomposition problem) and prove that the solving the new convex relaxation problems is guaranteed to achieve the globally optimal solutions of their original nonconvex ...

Li, Qiuwei — Colorado School of Mines

Robust Adaptive Machine Learning Algorithms for Distributed Signal Processing

Distributed networks comprising a large number of nodes, e.g., Wireless Sensor Networks, Personal Computers (PC’s), laptops, smart phones, etc., which cooperate with each other in order to reach a common goal, constitute a promising technology for several applications. Typical examples include: distributed environmental monitoring, acoustic source localization, power spectrum estimation, etc. Sophisticated cooperation mechanisms can significantly benefit the learning process, through which the nodes achieve their common objective. In this dissertation, the problem of adaptive learning in distributed networks is studied, focusing on the task of distributed estimation. A set of nodes sense information related to certain parameters and the estimation of these parameters constitutes the goal. Towards this direction, nodes exploit locally sensed measurements as well as information springing from interactions with other nodes of the network. Throughout this dissertation, the cooperation among the nodes follows the diffusion optimization ...

Chouvardas, Symeon — National and Kapodistrian University of Athens

On some aspects of inverse problems in image processing

This work is concerned with two image-processing problems, image deconvolution with incomplete observations and data fusion of spectral images, and with some of the algorithms that are used to solve these and related problems. In image-deconvolution problems, the diagonalization of the blurring operator by means of the discrete Fourier transform usually yields very large speedups. When there are incomplete observations (e.g., in the case of unknown boundaries), standard deconvolution techniques normally involve non-diagonalizable operators, resulting in rather slow methods, or, otherwise, use inexact convolution models, resulting in the occurrence of artifacts in the enhanced images. We propose a new deconvolution framework for images with incomplete observations that allows one to work with diagonalizable convolution operators, and therefore is very fast. The framework is also an efficient, high-quality alternative to existing methods of dealing with the image boundaries, such as edge ...

Simões, Miguel — Universidade de Lisboa, Instituto Superior Técnico & Université Grenoble Alpes

Adaptive Algorithms and Variable Structures for Distributed Estimation

The analysis and design of new non-centralized learning algorithms for potential application in distributed adaptive estimation is the focus of this thesis. Such algorithms should be designed to have low processing requirement and to need minimal communication between the nodes which would form a distributed network. They ought, moreover, to have acceptable performance when the nodal input measurements are coloured and the environment is dynamic. Least mean square (LMS) and recursive least squares (RLS) type incremental distributed adaptive learning algorithms are first introduced on the basis of a Hamiltonian cycle through all of the nodes of a distributed network. These schemes require each node to communicate only with one of its neighbours during the learning process. An original steady-steady performance analysis of the incremental LMS algorithm is performed by exploiting a weighted spatial-temporal energy conservation formulation. This analysis confirms that ...

Li, Leilei — Loughborough University

Distributed Signal Processing Algorithms for Wireless Networks

Distributed signal processing algorithms have become a key approach for statistical inference in wireless networks and applications such as wireless sensor networks and smart grids. It is well known that distributed processing techniques deal with the extraction of information from data collected at nodes that are distributed over a geographic area. In this context, for each specific node, a set of neighbor nodes collect their local information and transmit the estimates to a specific node. Then, each specific node combines the collected information together with its local estimate to generate an improved estimate. In this thesis, novel distributed cooperative algorithms for inference in ad hoc, wireless sensor networks and smart grids are investigated. Low-complexity and effective algorithms to perform statistical inference in a distributed way are devised. A number of innovative approaches for dealing with node failures, compression of data ...

Xu, Songcen — University of York

First-order Convex Optimization Methods for Signal and Image Processing

In this thesis we investigate the use of first-order convex optimization methods applied to problems in signal and image processing. First we make a general introduction to convex optimization, first-order methods and their iteration complexity. Then we look at different techniques, which can be used with first-order methods such as smoothing, Lagrange multipliers and proximal gradient methods. We continue by presenting different applications of convex optimization and notable convex formulations with an emphasis on inverse problems and sparse signal processing. We also describe the multiple-description problem. We finally present the contributions of the thesis. The remaining parts of the thesis consist of five research papers. The first paper addresses non-smooth first-order convex optimization and the trade-off between accuracy and smoothness of the approximating smooth function. The second and third papers concern discrete linear inverse problems and reliable numerical reconstruction software. ...

Jensen, Tobias Lindstrøm — Aalborg University

Self-organized Femtocells: a Time Difference Learning Approach

The use model of mobile networks has drastically changed in recent years. Next generation devices and new applications have made the availability of high quality wireless data everywhere a necessity for mobile users. Thus, cellular networks must be highly improved in terms of coverage and capacity. Networks that include smart entities and functionalities, and that allow to fulfil all the mobile networks’ new requirements are called heterogeneous networks. One key component in heterogeneous networks is femtocells. Femtocells are low range, low power mobile base stations deployed by the end consumers, which underlay the macrocell system and provide a solution to the problem of indoor coverage for mobile communications. Femtocells can reuse the radio spectrum and, thereby, they allow increasing the spectral efficiency. Moreover, under appropriate algorithms for interference control, they give a viable alternative to the problem of spectrum static ...

A. Galindo-Serrano — Centre Tecnològic de Telecomuniacions de Catalunya (CTTC)

Adaptive Noise Cancelation in Speech Signals

Today, adaptive algorithms represent one of the most frequently used computational tools for the processing of digital speech signals. This work investigates and analyzes the properties of adaptive algorithms in speech communication applications where rigorous conditions apply, such as noise and echo cancelation. Like other theses in this field do, it tries to tackle the ever-lasting problem of computational complexity vs. rate of convergence. It introduces some new adaptive methods that stem from the existing algorithms as well as a novel concept which has been entitled Optimal Step-Size (OSS). In the first part of the thesis we investigate some well-known, widely used adaptive techniques such as the Normalized Least Mean Squares (NLMS) and the Recursive Least Mean Squares (RLS). In spite of the fact that the NLMS and the RLS belong to the "simplest" principles, as far as complexity is ...

Malenovsky, Vladimir — Department of Telecommunications, Brno University of Technology, Czech Republic

The current layout is optimized for **mobile
phones**. Page previews, thumbnails, and full abstracts
will remain hidden until the browser window grows in width.

The current layout is optimized for **tablet
devices**. Page previews and some thumbnails will remain
hidden until the browser window grows in width.