In this thesis we investigate centralized and distributed variants of sparse Bayesian learning (SBL), an effective probabilistic regression method used in machine learning. Since inference in an SBL model is not tractable in closed form, approximations are needed. We focus on the variational Bayesian approximation, as opposed to others used in the literature, for three reasons: First, it is a flexible general framework for approximate Bayesian inference that estimates probability densities including point estimates as a special case. Second, it has guaranteed convergence properties. And third, it is a deterministic approximation concept that is even applicable for high dimensional problems where non-deterministic sampling methods may be prohibitive. We resolve some inconsistencies in the literature involved in other SBL approximation techniques with regard to a proper Bayesian treatment and the incorporation of a very desired property, namely scale invariance. More specifically, ...

This dissertation details two classes of cooperative localization methods for wireless networks in mixed line-of-sight and non-line-of-sight (LOS/NLOS) environments. The classes of methods depend on the amount of prior knowledge available. The methods used for both classes are based on the assumptions in practical localization environments that neither NLOS identification nor experimental campaigns are affordable. Two major contributions are, first, in methods that provide satisfactory localization accuracy whilst relaxing the requirement on statistical knowledge about the measurement model. Second, in methods that provide significantly improved localization performance without the requirement of good initialization. In the first half of the dissertation, cooperative localization using received signal strength (RSS) measurements in homogeneous mixed LOS/NLOS environments is considered for the case where the key model parameter, the path loss exponent, is unknown. The approach taken is to model the positions and the path ...

Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society. Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change. In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions. The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions. To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution. These methods enjoy well-understood statistical properties but are often computational prohibitive to employ. The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo ...

Wireless networks and power distribution grids are experiencing increasing demands on their efficiency and reliability. Judicious methods for allocating scarce resources such as power and bandwidth are of paramount importance. As a result, nonlinear optimization and signal processing tools have been incorporated into the design of contemporary networks. This thesis develops schemes for efficient resource allocation (RA) in such dynamic networks, with an emphasis in stochasticity, which is accounted for in the problem formulation as well as in the algorithms and schemes to solve those problems. Stochastic optimization and decomposition techniques are investigated to develop low-complexity algorithms with specific applications in cross-layer design of wireless communications, cognitive radio (CR) networks and smart power distribution systems. The costs and constraints on the availability of network resources, together with diverse quality of service (QoS) requirements, render network design, management, and operation challenging ...

Hyperspectral (HS) imaging, which consists of acquiring a same scene in several hundreds of contiguous spectral bands (a three dimensional data cube), has opened a new range of relevant applications, such as target detection [MS02], classification [C.-03] and spectral unmixing [BDPD+12]. However, while HS sensors provide abundant spectral information, their spatial resolution is generally more limited. Thus, fusing the HS image with other highly resolved images of the same scene, such as multispectral (MS) or panchromatic (PAN) images is an interesting problem. The problem of fusing a high spectral and low spatial resolution image with an auxiliary image of higher spatial but lower spectral resolution, also known as multi-resolution image fusion, has been explored for many years [AMV+11]. From an application point of view, this problem is also important as motivated by recent national programs, e.g., the Japanese next-generation space-borne ...

In this dissertation we focus on statistical signal processing for Data Fusion, with a particular focus on wireless sensor networks. Six topics are studied: (i) Data Fusion for classification under model uncertainty; (ii) Decision Fusion over coherent MIMO channels; (iii) Performance analysis of Maximum Ratio Combining in MIMO decision fusion; (iv) Decision Fusion over non-coherent MIMO channels; (v) Decision Fusion for distributed classification of multiple targets; (vi) Data Fusion for inverse localization problems, with application to wideband passive sonar platform estimation. The first topic of this thesis addresses the problem of lack of knowledge of the prior distribution in classification problems that operate on small data sets that may make the application of Bayes' rule questionable. Uniform or arbitrary priors may provide classification answers that, even in simple examples, may end up contradicting our common sense about the problem. Entropic ...

In this thesis, a general solution to the component separation problem in images is introduced. Unlike most existing works, the spatial dependencies of images are modelled in the separation process with the use of Markov random fields (MRFs). In the MRFs model, Cauchy density is used for the gradient images. We provide a general Bayesian framework for the estimation of the parameters of this model. Due to the intractability of the problem we resort to numerical solutions for the joint maximization of the a posteriori distribution of the sources, the mixing matrix and the noise variances. For numerical solution, four different methods are proposed. In first method, the difficulty of working analytically with general Gibbs distributions of MRF is overcome by using an approximate density. In this approach, the Gibbs distribution is modelled by the product of directional Gaussians. The ...

Every day we share our personal information through digital systems which are constantly exposed to threats. For this reason, security-oriented disciplines of signal processing have received increasing attention in the last decades: multimedia forensics, digital watermarking, biometrics, network monitoring, steganography and steganalysis are just a few examples. Even though each of these elds has its own peculiarities, they all have to deal with a common problem: the presence of one or more adversaries aiming at making the system fail. Adversarial Signal Processing lays the basis of a general theory that takes into account the impact that the presence of an adversary has on the design of effective signal processing tools. By focusing on the application side of Adversarial Signal Processing, namely adversarial information fusion in distributed sensor networks, and adopting a game-theoretic approach, this thesis contributes to the above mission ...

In this thesis we focus on the development of energy-efficient adaptive algorithms for Wireless Sensor Networks. Its contributions can be arranged in two main lines. Firstly, we focus on the efficient management of energy resources in WSNs equipped with finite-size batteries and energy-harvesting devices. To that end, we propose a censoring scheme by which the nodes are able to decide if a message transmission is worthy or not given their energetic condition. In order to do so, we model the system using a Markov Decision Process and use this model to derive optimal policies. Later, these policies are analyzed in simplified scenarios in order to get insights of their features. Finally, using Stochastic Approximation, we develop low-complexity censoring algorithms that approximate the optimal policy, with less computational complexity and faster convergence speed than other approaches such as Q-learning. Secondly, we ...

Sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) methods provide computational tools for systematic inference and learning in complex dynamical systems, such as nonlinear and non-Gaussian state-space models. This thesis builds upon several methodological advances within these classes of Monte Carlo methods. Particular emphasis is placed on the combination of SMC and MCMC in so called particle MCMC algorithms. These algorithms rely on SMC for generating samples from the often highly autocorrelated state-trajectory. A specific particle MCMC algorithm, referred to as particle Gibbs with ancestor sampling (PGAS), is suggested. By making use of backward sampling ideas, albeit implemented in a forward-only fashion, PGAS enjoys good mixing even when using seemingly few particles in the underlying SMC sampler. This results in a computationally competitive particle MCMC algorithm. As illustrated in this thesis, PGAS is a useful tool for both ...

This thesis presents a comprehensive example framework on how current multiple model state estimation algorithms for jump Markov linear systems can be improved. The possible improvements are categorized as: -Design of multiple model state estimation algorithms using new criteria. -Improvements obtained using existing multiple model state estimation algorithms. In the first category, risk-sensitive estimation is proposed for jump Markov linear systems. Two types of cost functions namely, the instantaneous and cumulative cost functions related with risk-sensitive estimation are examined and for each one, the corresponding multiple model estate estimation algorithm is derived. For the cumulative cost function, the derivation involves the reference probability method where one defines and uses a new probability measure under which the involved processes has independence properties. The performance of the proposed risk-sensitive filters are illustrated and compared with conventional algorithms using simulations. The thesis addresses ...

The discovery of novel gene regulatory processes improves the understanding of cell phenotypic responses to external stimuli for many biological applications, such as medicine, environment or biotechnologies. To this purpose, transcriptomic data are generated and analyzed from DNA microarrays or more recently RNAseq experiments. They consist in genetic expression level sequences obtained for all genes of a studied organism placed in different living conditions. From these data, gene regulation mechanisms can be recovered by revealing topological links encoded in graphs. In regulatory graphs, nodes correspond to genes. A link between two nodes is identified if a regulation relationship exists between the two corresponding genes. Such networks are called Gene Regulatory Networks (GRNs). Their construction as well as their analysis remain challenging despite the large number of available inference methods. In this thesis, we propose to address this network inference problem ...

Distributed networks comprising a large number of nodes, e.g., Wireless Sensor Networks, Personal Computers (PC’s), laptops, smart phones, etc., which cooperate with each other in order to reach a common goal, constitute a promising technology for several applications. Typical examples include: distributed environmental monitoring, acoustic source localization, power spectrum estimation, etc. Sophisticated cooperation mechanisms can significantly benefit the learning process, through which the nodes achieve their common objective. In this dissertation, the problem of adaptive learning in distributed networks is studied, focusing on the task of distributed estimation. A set of nodes sense information related to certain parameters and the estimation of these parameters constitutes the goal. Towards this direction, nodes exploit locally sensed measurements as well as information springing from interactions with other nodes of the network. Throughout this dissertation, the cooperation among the nodes follows the diffusion optimization ...

Compressed sensing (CS) is a recently introduced signal acquisition framework that goes against the traditional Nyquist sampling paradigm. CS demonstrates that a sparse, or compressible, signal can be acquired using a low rate acquisition process. Since noise is always present in practical data acquisition systems, sensing and reconstruction methods are developed assuming a Gaussian (light-tailed) model for the corrupting noise. However, when the underlying signal and/or the measurements are corrupted by impulsive noise, commonly employed linear sampling operators, coupled with Gaussian-derived reconstruction algorithms, fail to recover a close approximation of the signal. This dissertation develops robust sampling and reconstruction methods for sparse signals in the presence of impulsive noise. To achieve this objective, we make use of robust statistics theory to develop appropriate methods addressing the problem of impulsive noise in CS systems. We develop a generalized Cauchy distribution (GCD) ...

The first part of this dissertation considers distributed learning problems over networked agents. The general objective of distributed adaptation and learning is the solution of global, stochastic optimization problems through localized interactions and without information about the statistical properties of the data. Regularization is a useful technique to encourage or enforce structural properties on the resulting solution, such as sparsity or constraints. A substantial number of regularizers are inherently non-smooth, while many cost functions are differentiable. We propose distributed and adaptive strategies that are able to minimize aggregate sums of objectives. In doing so, we exploit the structure of the individual objectives as sums of differentiable costs and non-differentiable regularizers. The resulting algorithms are adaptive in nature and able to continuously track drifts in the problem; their recursions, however, are subject to persistent perturbations arising from the stochastic nature of ...